Phase 1 – Identification and analysis of theories of change

Although Theories of Change practice is still a developing field, the generally accepted starting point is to 'surface' the theory of change collaboratively with relevant stakeholders by initially identifying the 'vision' or long-term outcome and then setting out the pathway to be taken to reach this goal (Anderson, 2005;

Connell and Kubisch, 1998). With each community organisation, the aim was to undertake these first steps through one or two participative workshops.

Preparation for this included examination of any relevant documentation from the organisation (e.g. annual reports), to enable the workshop(s) to be appropriately structured and focused. Although the workshop(s) were tailored to each

organisation to some degree, the basic structure was designed with three elements:

Chapter 6

181

 Identification of the aims of the organisation – combining both the specific targets of the organisation and its vision for the community as a whole

 Examination of past achievements of the organisation – exploring the pathways by which these achievements had been delivered

 Surfacing and exploration of theories of change for future targets, drawing together the long-term vision and the ideas about pathways to change relating to previous achievements.

Thus the intention was to use a process of visioning ultimate aims, combined with reflection on past activities and achievements, to delineate theories of change for future activity which could be tested through Phases 2 and 3 of the research.

Whilst the development of local ToC models and the examination of their

implementation was intended to address Research Questions 2 and 3 in particular, it was important in this first phase of the research to focus on each organisation’s approach and avoid skewing these local models to fit the prior analysis of national policy assumptions from Chapter 4. Hence, although I had already done the analysis around responsibilisation and the specific national policy assumptions, I attempted to facilitate the articulation of local ToCs without introducing such ideas into the discussions or the models.

The discursive process of Phase 1 was also intended to examine the assumptions underlying the emergent theories of change, utilising the key questions of

plausibility, doability and testability (Connell and Kubisch, 1998: 19; Anderson, 2005: 25), and to start to identify appropriate indicators for outcomes within each model, which would be measured in Phase 2. This element of Theories of Change methodology serves a dual purpose. Firstly, by checking that the model is logically plausible, that the stages and goals involved are achievable, and that the different elements can be realistically measured, the aim is to ensure that it will serve a useful evaluative purpose. As Weiss (2007: 70) argues, if the model makes sense and the evaluation data shows both that outcomes have been achieved and the micro-steps between inputs and outcomes have occurred, then we can plausibly argue that we have shown causal attribution 'for all practical purposes'. And secondly, the process of questioning the assumptions within a theory of change is

Chapter 6

182 in itself a useful developmental tool for organisations, facilitating reflection on practice. From the perspective of the research project, the process of identifying indicators was particularly important not merely to enable the testing of the individual ToCs, but also to provide data that might assist with the RE analysis of the hypothesised mechanisms from Chapter 5. Again, it was important not to skew the choices of indicators away from those that would be of most benefit for the participant organisations, but discussion around the measurable indicators did allow for the inclusion of some additional items which would provide useful data relating to the mechanisms.

The issue of who was involved in the workshop(s) was a matter for negotiation with each organisation, but in general the management committee of each organisation was key, with staff participating alongside if felt appropriate. Follow-up interviews with key individuals who were not involved in the workshop(s) were conducted where necessary, to explore their perspectives on the identified

theories of change. One of the challenges of facilitating the articulation of theories of change is managing situations where there are multiple theories operating within an organisation or initiative (Mackenzie and Blamey, 2005: 153-4). The key at this stage of the process is not necessarily to seek absolute consensus, but to ensure that different pathways to change can be tested, in order to adjudicate between them to some degree (Connell and Kubisch, 1998: 29-31). In practice, however, this proved to be a very minor issue within the research, perhaps reflecting the extent to which the participant organisations, as voluntary bodies, had already had to deal with differences of approach in order to work effectively together. More significant issues were encountered with variable boundaries as to who should be involved in these Phase 1 workshops, with some organisations being represented by only a small sub-group of their membership, whilst one organisation involved a wide range of people beyond the committee. The

Armitshore case study also raised a particular challenge at this stage, since the fluid membership and limited organisation of the Neighbourhood Action Groups made it impossible to undertake workshops with the NAGs themselves. Hence the research for Phases 1 and 2 was primarily conducted through a workshop with the Localities Team and observations of NAG meetings. These issues are explored further in Chapter 10, which addresses the methodological research questions.

Chapter 6

183 Alongside the workshops with community organisations in this first phase of the research, a number of interviews were undertaken with relevant individuals from the local authorities to explore the local ToCs from the perspective of the authority.

As noted earlier, it would not be feasible within this project to examine the

community participation ToCs of the authorities themselves, as these would likely be diverse and more complicated to surface than those of the community

organisations. Hence these interviews explored the authority's understanding of the role of each organisation, with the aim of enhancing the analysis of interactions between local and national ToCs, addressing Research Questions 2 and 3. In particular, they aimed to address some of the national policy assumptions identified in Chapter 4 regarding the power dynamics within community

participation processes and the role of the state, exploring the different ways in which community participation might be employed by agencies to hold on to or share power (Croft and Beresford, 1996: 192).

Table 6.4 below provides a summary of the workshops, interviews and observations relating to each participant organisation. These were primarily focused on the Phase 1 process of elucidating the local ToC, as outlined above, although in some instances elements of Phase 2 data collection were also included, particularly in the local authority interviews. The full detail of data sources for all three phases of the research is provided in Appendix D.

Table 6.4 – Summary of Phase 1 workshops and interviews

Case study area

Participant organisation workshops and interviews

Local authority interviews

Trottside 2 workshops (3 PC members, 7 community members)

3 informal interviews (2 PC members)

3 interviews (2 officers, 1 Councillor)

Hoyfield 1 workshop (3 board members) 4 informal interviews (Centre Manager)

1 interview (officer)

Armitshore 3 observations (NAG meetings) 1 workshop (7 Localities Team members)

3 informal discussions after meetings (2 Localities Team members)

Dowsett 2 workshops (12-15 CC members) 2 informal interviews (CC Chair) 12 observations (CC meetings)

1 interview (officer)

Chapter 6

Ooley 2 workshops (4-6 board members) 2 informal interviews (2 board members)

1 observation (ODT AGM)

1 interview (officer)

1 observation (Development Steering Group meeting) Cavendish 1 workshop (3 board members, 7

volunteers/service users) 2 informal interviews (CWL manager)

3 observations (2x large events run by CWL, 1 board meeting)

1 discussion (4 staff members)

1 interview (officer)