What do we mean by policy?

Much of the evaluation literature refers to social programmes as much as to policies, reflecting the tendency for specific programmes to be subject to evaluation more frequently than broad policy areas or ongoing public service provision. For the purposes of simplicity within this section, I shall use the terms interchangeably, although there is significant debate regarding the links between policies and programmes, as I shall discuss later in the chapter.

4 Fankle - a Scots word meaning ‘tangle’ or ‘confusion’. Not to be confused with ‘bourach’, which is an even messier, irresolvable fankle. The reader can judge whether the right term has been employed here.

35 As a starting point, Chen provides a usefully straightforward definition:

"A social intervention or programme is the purposive and organised effort to intervene in an ongoing social process for the purpose of solving a problem or providing a service." (Chen, 1990: 39)

However, as Weiss suggests, this apparently simple description belies a much more complicated reality:

"Social programmes are complex undertakings. They are an amalgam of dreams and personalities, rooms and theories, paper clips and organisational structure. clients and activities, budgets and photocopies, and great intentions" (Weiss, 1998: 48)

Hence understanding and evaluating policies and programmes is inherently challenging because of their complexity (not to mention the difficulty of getting a handle on dreams and personalities). Rather than baulking at this hurdle, however, it is useful to examine different aspects of complexity within policy. In this respect it is useful to draw on Glouberman and Zimmerman’s (2002) distinctions between simple, complicated and complex, which they illustrate with the examples of baking a cake, sending a rocket to the moon and raising a child. In simple processes, like cake-baking, following a recipe without substantial expertise will reliably produce a reasonably good result through a linear process. In complicated processes, like rocket science, much greater expertise and coordination is

required, but the process is still relatively linear and experience can produce a reasonable certainty of outcome. Complicated processes are not simply a

combination of multiple simple processes, but they are predictable enough to be generalizable. Finally, in complex situations, expertise can be useful, but the emergent, adaptive, non-linear nature of the process and the uniqueness of each case mean that outcomes can never be confidently predicted.

Employing these distinctions, it is useful to examine six inter-related dimensions of policies, to identify elements of simplicity, complicatedness or complexity.

36 Firstly, policies operate at different scales. Whilst the relation is not

straightforward, policies which apply across larger areas, over longer timescales, affecting larger numbers of people and/or organisations, are clearly complicated and will often contain significant complexity. Thus, national policies such as Localism or Community Empowerment which are multi-faceted, long-term and necessitate the involvement of multiple bodies at different levels are obviously more complex than a single, local project.

The second dimension is what Kubisch et al (1998: 4) call 'vertical complexity', referring to the range of targets for change at different levels. Whilst some policies may aim for change only at the individual level (though it is difficult to find clear examples), many policy interventions, such as the 'comprehensive community initiatives' that primarily concern Kubisch et al and which are closer to the subject of this study, aim to achieve change at individual, community, organisational, and even policy levels. The possibility of interactions between outcomes targets, such that individual outcomes may affect community processes and vice versa, suggest that such initiatives need to be seen as vertically complex. Moreover, for many policies, there is an additional element of complexity in terms of outcome targets, because of a lack of consistent and clear objectives (Sabatier, 1997: 278). This is particularly true of those involving community participation, where targets are to some extent emergent, evolving over time, as well as being significantly contested and ambiguous.

Thirdly, policies vary significantly in terms of the level of process complexity. As Pawson (2006: 28) expresses it, for many policies, "Intervention chains are long and thickly populated", as well as often being non-linear. Again, community

participation policies clearly contain significant complexity in this respect, since the links between policy levers such as legislation and outcomes in communities are characterised by multiple stages and potential for both feedback and unintended consequences.

The fourth dimension is the degree of organisational complexity. Whilst some policies involve interactions between just one organisation and individual members of the public, this is relatively rare. Indeed, as Asthana et al (2002: 780) point out, there is often an emphasis on partnership working in social policy, on the basis

37 that complex social problems require multi-faceted responses. Again, community participation policies almost universally involve multiple organisations, including a range of public agencies and community organisations. Moreover, these

organisations and the relations between them may be emergent through the process, suggesting a significant degree of complexity (Funnell and Rogers, 2011).

Fifthly, it is important to consider the degree of agency involved in any policy, since greater agency increases the adaptive, non-linear nature of processes.

Moreover, as highlighted in the previous section, agency is central to

considerations of causality. As Lipsky (1997) identified, policies are shaped in practice by bureaucrats at the 'street-level' as well as the politicians and managers who are conventionally seen as 'policy makers'. Similarly, Sabatier (1997) points to the value of exploring how policies are developed from the bottom up, as much as from the top down. Clearly community participation policy has a particular focus on agency, since it focuses on activating and engaging communities.

Finally, the sixth dimension relates to the fact that policies are inherently open systems. As Minogue (1997: 11) argues, we cannot study individual decisions or policies in isolation, since every policy interacts with wider policies, networks and social systems. In Pawson's words, "social interventions are always complex systems thrust amidst complex systems" (2006:35, italics in original). Again, community participation processes are clearly complex in this respect, since communities interact with each other, with agencies, with other policies and with wider social structures.

Thus it seems clear that community participation policies are significantly complex in relation to all six dimensions outlined above. As Funnell and Rogers suggest, this creates substantial challenges for evaluation:

“Dynamic and emergent interventions present a challenge to conventional linear processes of developing an evaluation,

implementing it, and reporting the findings. Dynamic interventions change substantially over time, and their specific impacts cannot always be identified in advance.” (Funnell and Rogers, 2011: 79)

38 However, the purpose of laying out these dimensions of complexity is not to

generate despair, but rather to highlight the challenges in order to inform the choice of methodology. Thus the next step is to examine what it is that we want to know within an evaluation of community participation policy and practice and therefore which methodology might be most appropriate.