Both RE and NPT are theoretically informed approaches that can help researchers to further investigate, develop and refine middle-range theories (MRTs) to explain specific parts of programmes and interventions.
Merton defines MRTs as‘theories that lie between the minor but necessary working hypotheses and the
all-inclusive systematic efforts to develop a unified theory that will explain all the observed uniformities of
social behaviour, social organisation and social change’ (p. 39).120MRTs are thus sufficiently abstract to be
applied to differing spheres of social behaviour and structure, but do not offer a set of general laws about these at societal level. Specifying the range of the theory is important here. The limited scope, conceptual range and claims of MRTs are what make them practically workable in analysing practice; both approaches (RE and NPT) can be applied across different types of social programmes at different scales.
The bodies of literature on empirical applications of both these approaches are relatively recent,112
especially NPT; its third and final development phase was reported only in 2009.121
Realist evaluation is rooted in the philosophy of science situated between extremes of positivism and
relativism, known as realism.115,122Realism understands the world as an open system, with structures and
layers that interact to form mechanisms and contexts. RE research is concerned with the identification of
underlying causal mechanisms, how they work, under what conditions and for whom.115,123The approach
employs three key terms, for which definitions are provided in Box 2.
BOX 2 Definitions of terms of RE as applied to PPI in research (adapted from Pawson and Tilley115
and Jagosh et al.99
)
Context
Context often pertains to the‘backdrop’ of programmes (PPI in research): those conditions within which programmes are introduced that are relevant to operating the programme mechanisms (e.g. the research arena, previous working relationships or funding body). As these conditions change over time, the context may reflect aspects of those changes while the programme is implemented. Context can be broadly understood as any condition that triggers and/or modifies the mechanism.
Mechanism
Mechanisms are the unit of analysis of RE. Mechanisms describe what it is about programmes and interventions (PPI in research) that bring about any effects (outcomes). Mechanisms are often hidden, as with a clock’s workings; they cannot be seen but drive the patterned movements of the clock’s hands. Mechanisms are linked to, but not synonymous with, the programme’s strategies (e.g. a strategy may be an intended plan of action for PPI in research, whereas a mechanism involves the researchers’ and PPI representatives’ reaction or response to the intentional offer of incentives or resources). Identifying the mechanisms (e.g. the design of PPI roles within research projects) advances the synthesis beyond describing what happened to theorising why it happened, for whom and under what circumstances.
Outcomes
Outcomes are either intended or unintended consequences of programmes, resulting from the activation of different mechanisms in different contexts (e.g. if the research recruited more successfully because of PPI). Outcomes can be proximal, intermediate or final.
Because causal mechanisms are always embedded within particular contexts and social processes, their complex relationship with the effect of context on their operationalisation and outcome needs to be
understood. Pawson and Tilley115describe this as: context (C)+ mechanism (M) = outcome (O). Rather
than identifying simple cause-and-effect relationships, RE is concerned with finding out about what
mechanisms work, in what conditions, why, to produce which outcomes and for whom.124RE does this
through testing and refining configurations of context, mechanism and outcome (CMO) within analysis of data, as described in Box 3.
Although Pawson and Tilley did not provide a set of methodological rules for RE, and these are still much
debated,112,125they nonetheless suggest some principles for guiding evaluators. These include the
identification of mechanisms and testing/refining of CMO configurations; stakeholder involvement and engagement; and a generative conception of causality (i.e. an explanation not of variables that are
inter-related to one another but rather of how they are associated).124
In practice, REs start with a MRT and end with a refined MRT, following four main stages as described by
Pawson and Tilley.114The MRT can be formulated on the basis of existing theory, past experience and
previous evaluations or research studies. The result is discussed with the stakeholders and finally results in the MRT that will be tested. The field study is then designed in relation to the MRT: the design, data collection tools and analysis tools are developed to enable testing the elements of the MRT. The outcomes of the fourth stage should lead to policy advice and recommendations. The application of these four stages in this research is shown in Figure 2.
• Literature review • Previous research • Stakeholder discussion Selection of programme theories and propositions
• Map outcomes across cases
• Develop CMO configurations • Identify salient actions that could explain outcome patterns
Test propositions • Scoping and survey
• Case studies Data collection • Stakeholder discussion on CMO configurations • Refinement of programme theories • Confirmation of transferable salient actions Refinement BOX 3 Definition of CMO configurations (adapted from Pawson and Tilley115
and Jagosh et al.99)
Context, mechanism and outcome configurations
Context, mechanism and outcome configuring is a heuristic tool used to generate causative explanations pertaining to outcomes in the observed data. The process draws out and reflects on the relationship of CMO of interest in a particular programme. A CMO configuration may pertain either to the whole programme or only to certain aspects. One CMO may be embedded in another or configured in a series (in which the outcome of one CMO becomes the context for the next in the chain of implementation steps). Configuring CMOs is a basis for generating and/or refining the theory that becomes the final product of the evaluation. RE thus develops and tests CMOs conjectures empirically.
Both a published literature review112and critical discussions in the applied literature cite several
methodological challenges associated with a RE approach, including deciding what constitutes a mechanism,127
and indeed differentiating between what is a mechanism and what is context. Barnes et al.,128for example,
also caution against interpreting context as a purely external factor, arguing that context is shaped by actors as much as it constrains their activities.