Using Methodological Approaches within Impact Studies

As discussed in the previous section, fair trade studies must be appropriately planned and should seek to address the following issues: the purpose of the assessment; the criteria and indicators to be used; stakeholder involvement in the impact assessment; the particular interventions to be assessed; the contextual factors to be included; and how the findings are to be fed into practice (Mayoux, 2012).

When analysing fair trade and developing a study to address these issues, certain tools are more appropriate than others. Paul (2005) discusses quantitative, qualitative and participatory methods in detail arguing that the use of quantitative methods cannot realistically be applied to an evaluation of fair trade. This is because, ideally, quantitative evaluations aim to incorporate what outcomes would be without intervention, enabling a

“with-and-without” comparison between two sets or a “before-and-after” comparison where the test group and control group are the same. The problem with this approach is that “it is difficult to envisage a Fair Trade organisation being bent on evaluation to the point of conducting an experiment within a cooperative which entailed the random selection of beneficiaries” (Paul, 2005, p140). That is to say, difficulties emerge since randomisation requires the random division of eligible individuals into two groups: those

125

who receive the intervention and those who do not, creating practical and ethical problems.

Paul (2005) recommends the use of Rapid Appraisal Methods which have evolved over the last two decades. Rapid appraisal methods are quick, low-cost methods for gathering information, that lie somewhere along the continuum of data collection options ranging from informal short field visits & casual conversations to census, surveys, or experiments.

Bergeron (1999) explains that Rapid Appraisal Methods offer “a useful set of research and appraisal tools to obtain quickly information from local populations about their conditions and their needs” (Bergeron, 1999, p3). Thus, this approach will enable local people to plan alongside outsiders leading to appropriate interventions as well as an evaluation of the impact of interventions have had after they have been carried out.

While limitations are evident with these Rapid Appraisal Methods, such as the lack of random sampling and influence of the researcher’s judgement, they provide fast access to relevant information, enable flexibility and require little investment and few resources. The most common techniques used are: key informant interviews; direct and structured observation; and informal surveys.

A participatory approach to analysing fair trade is recommended in Paul (2005) and Mayoux’s (2001) papers. Paul recommends this should be carried out using Rapid Rural Appraisal (RRA). The RRA, a type of Rapid Appraisal Method, “uses a non-standard set of methods for collecting and analysing information, ranging from semi-structured interviews to analytical games. On this basis, miscellaneous methods of participative enquiry have been designed with a view to bridging the social and cognitive gap between a project’s

126

beneficiaries and its evaluators” (Paul, 2005, p141). Paul’s experience of implementing this technique involves the use of non-standard methods such as visual techniques including charts, illustrating the history of the community and highlighting changes that have occurred, in order to show advances in living conditions. The variety of techniques used in this method have allowed researchers to gain a better understanding of producers, to tackle power and gender issues and to see who benefits the most from fair trade, employees or smallholders.

Also applying RRA, Bacon (2005) undertook an impact study in Nicaragua using a survey containing structured closed ended interview questions. In addition, working with gender specialists he conducted ten focus groups separated by sex, using participants from the same list of farmers who participated in the survey to triangulate their answers. Interviews were also carried out with leaders of the cooperatives and professional staff.

Mayoux (2001) states it is important for fair trade impact assessments to include a range of political, economic, social and environmental criteria and to involve different stakeholders who may all have differing interests in the interventions and outcomes. It is argued, given the commitment of fair trade to help the most disadvantaged, their views and interests “must be given at least equal weight in terms of selection criteria for assessment representation in the impact assessment process and analysis of the types of impact and the practical implications” (Mayoux, 2001, p13). Moreover, in order to allow for credible recommendations to be drawn, there must be a careful analysis of the particular type of intervention according to its aims, structure and the nature of the activities involved. The analysis of economic and socio-economic factors must be placed in a context that includes international, national and local markets for products, input and

127

labour, and incorporates the opportunities and constraints facing other private sector enterprises. Further, Mayoux states that, given the aims of fair trade, assessment should contribute to the building up of a sustainable and participatory monitoring evaluation system, including capacity building for producers themselves and, where possible, communities, to monitor and evaluate. There should also be a contribution to networking, learning and accountability between organisations, (Mayoux, 2012). Recommendations derived from discussions between various fair trade stakeholders can highlight areas of focus for future impact studies on fair trade. Such recommendations include how the benefits of fair trade can be increased, how fair trade markets can be expanded, and how the impact of fair trade on macro-level policy and the mainstream market i.e. conventional trade, might be increased (Mayoux, 2012).

Other tools which may be useful in an impact study are Social Impact Assessments (SIA).

These make use of several methods for collecting qualitative data, such as key informant interviews and targeted surveys, in order to examine how a given reform distributes the costs and benefits amongst stakeholders, (Paul, 2005). This approach makes use of several methods for collecting qualitative data such as key informant interviews and targeted surveys. Previous studies in Ghana (Jones and Bayley, 2000; NRET 2000) have used this method to show that the impact of fair trade was limited due to the combination of a weak cooperative labour force and an absence of related development projects (Paul, 2005).

Hence, it is argued, this method is useful in evaluating the context and assets required to reap the potential benefits from fair trade.

SIA studies of Coocafé in Costa Rica (Ronchi, 2002) involved field interviews on three levels:

the secondary level of Coocafé; the cooperative; and individual producers. Selection of

128

cooperatives was not random but aimed to build a representative sample in terms of geography, size and experience of the fair trade market.

Table 3.1 below summarises some of the methods used in previous impact studies. As is evident from the preceding discussion, there are a range of approaches to the evaluation of fair trade. It is clear that the process of research design need to be sensitive of context and stakeholder interests. Accordingly, evidence and the recommendations from the literature are used to inform the methodology employed in this study.

Table 3.1 Methods employed in previous impact studies

Study Cooperative and location Methodological approach Jones and Bayley (2000).

Traidcraft (2000) AMKA (Tanzania) and Just X (South Africa)

129

Ronchi (2002) Costa Rica - involves field interviews

on three levels: the

Bacon (2005) Nicaragua - RRA approach to survey

- Closed ended questions

130

- Group 1 is organic farmers, group 2 farmers under conversion, group 3 is fruit farmers and group 4 have no affiliation to the fair trade cooperative (30 per group)