Pilot interviews and questionnaire design

Before conducting a larger number of interviews there is an option of conducting pilot interviews in qualitative research to check all questions and topics are understood (Krueger and Casey, 2009). For overseas fieldwork, and in cases where the

interviewer has no experience of a country or culture (as was the case in my research), Sampson (2004: 399) identifies a substantial benefit of conducting pilot interviews in discovering that ‘pilots are invaluable as introductions to unknown worlds’. After

106 Refer to Appendix C for an example of how we introduced the project and each interview session.

conducting a small number of interviews it is then possible to return, reflect, revise, and rewrite or add questions based on the responses received and unexpected issues uncovered. In this process richness can be added to the material obtained once a deeper understanding of agendas, themes, and contextual nuances are gained.

Before organising pilot interviews I constructed a draft, semi-structured questionnaire.

Bryman (2008) discusses two types of qualitative interview; unstructured and semi-structured. In an unstructured interview the researcher may start with just one question and thereafter the interview is dictated by the subsequent conversation.

Semi-structured interviews start with a longer list of questions as an interview schedule.

Depending on a particular circumstance, the interview may not strictly follow that schedule; questions can be moved forwards or backwards, and there is the opportunity to probe and explore key issues that the interviewer may not have previously

considered. The epistemological utility of semi-structured interviews is summarised by Roche (1999: 108) in stating that ‘simply talking, and listening, to people is probably the most common and useful way of assessing impact’. Semi-structured interviews also allow the possibility of drawing out sensitive information on relations with others, impacts, and responses to shocks such as flooding or drought that may affect livelihoods.

In my research, consistent with both grounded theory and providing space for subaltern perspectives, my semi-structured questionnaire included open-ended questions that enabled participants to provide spontaneous, substantial feedback to such questioning. The quality of the feedback received was heightened by re-confirming understanding of any questions that were unclear. The semi-structured interview format also proved useful as a level of consistency when at times interviews were conducted by more than one person, as I explain below.

To begin construction of a questionnaire, referring to Table 5.4, I listed a series of questions under each section aimed at eliciting experiences, understandings, and expectations of fair trade. Consistent with a grounded theory approach, I maintained a format of open-ended questions to permit respondents to answer in their own terms (Bryman, 2008; Flick, 2009). My hope was that this would help capture

un-anticipated themes from lived experiences of participants.

As part of that process I recruited a research assistant to conduct interviews at KCGL in Chichewa (the participants’ first language). In order to provide space for voices to be heard through a conversational interview process it was necessary to seek some research assistance as neither I nor the majority of potential research participants shared sufficient common language to conduct such an interview. Also, interviewers sharing the same nationality as the participants receive greater authority due to the greater familiarity (Edwards, 1998).

I first recruited a graduate of the University of Malawi in Zomba, Monica Mmanga, who had experience of working for consultants on international development research projects and, therefore, possessed valuable skills and knowledge of interviewing people in low income districts of Malawi. As a graduate with this experience she possessed a position of authority which in practice was respected throughout our engagements with research participants. Associated effective interpersonal and

communication skills helped to garner positive relations and aided in collecting richer material. Being able to afford to employ research assistants was a further positive contribution of the funding secured for my studentship and additional funding provided by Traidcraft.

As noted above, caution should be applied when working with assistants in the interview process as misunderstandings or misinterpretations may arise. To help overcome such concerns I inducted Monica into the research by explaining the background, aims, and nuanced aspects of my project. As part of this best practice proposed by Edwards (1998) we also agreed on the confidential nature of the

interviews, our respective roles, and parameters for our work such as rates of pay and other conditions including travel and accommodation costs.¹⁰⁷

To conduct interviews in private we were provided with access to a spare room to the side of KCGLs main office block. Over a course of three days at Kasinthula we interviewed six farmers and nine employees (including two seasonal employees) as our sample for pilot interviews with each interview lasting up to one hour. These numbers were selected based on combined issues of time constraints, capturing

107 An example of an agreement that specified such details can be found in Appendix H. I drafted the document based upon local advice and the request of assistants.

perspectives of each category of Kasinthula producer, and limiting numbers to permit analysis of pilots before taking the time of too many people.

In preparation for interviews we agreed to make the hour as informal as possible by encouraging participants to talk freely and feel comfortable. We conducted all of the pilot interviews together so that we could both get first-hand experience of how the interviews and pilot questionnaire were being received. Monica asked questions in Chichewa and translated summaries of the responses to me as the interview

progressed. She made participants feel comfortable and we both answered their questions as fully as possible. The interviews were carried out using a triangular seating arrangement following best practice for such situations (Edwards, 1998).

At the end of each interview we consulted to make some amendments in light of our experience and drew upon Monica’s experience of conducting village based research.

We amended some questions in light of first pilots, re-ordered them to make the flow of the interview better, and removed some that we thought may be raising

expectations of direct action resulting from the interviews.

There were two lessons that I took from this initial interview experience that I reflected upon when similar instances arose in later interviewing. First, on a number of occasions participants viewed me as someone positioned to facilitate immediate change, with links to more donor aid funding, and who could liaise in local socio-political disputes by taking messages from one party to another. This follows dilemmas of cross-cultural research highlighted by Edwards (1998), Herod (1999), and Skelton (2001) that outsiderness and whiteness affect ways in which participants view and respond to you as a researcher.

In response to such messages, I re-iterated the purpose, and therefore the limitations, of what I could promise in my position. To do that I re-focussed discussions onto contributions to a wider fair trade knowledge base and stated that it was not my purpose to directly facilitate material benefits or further funding or support.¹⁰⁸ However, with a postcolonial influence in my methodology to present subaltern

108 This dilemma is also discussed by Punch (2001).

perspectives, I did emphasise that voices of all participants will be represented in my research dissemination. That dissemination is with the knowledge that I am

representing the perspectives of others and including my interpretations.

A second lesson we took from the pilots related to interview location, one aspect related to ethics in research highlighted by the ESRC (2010). In their experience, Hale and Opondo (2005) decided sometimes to interview workers at locations away from farms to offer the right to privacy and help in avoiding undue harm and breaches of confidentiality (ESRC, 2010; Flick, 2009; Fontana and Frey, 2008). After conducting pilot interviews we offered to interview some farmers and employees in their villages rather than just at the KCGL offices. This was in part to make arrangements more convenient for participants who less frequently visited the office block compared to others. It was also to offer neutral locations when appropriate to ensure participants were comfortable and confident in the situation. This issue arose on a few occasions with seasonal employees and members of the Fairtrade Premium Committee as I discuss later in this chapter and in Chapter Six.

By conducting the pilot interviews I allowed those being interviewed to participate in the research and questionnaire design by using what they said to determine the focus of the research and remaining material collection. After completing the pilot

interviews we reviewed all of the responses collected to look for interesting issues and themes emerging to guide the design of interview schedules and questionnaires for interviewing larger numbers of participants.

This review led to a redesign of my questionnaire. To test the new questionnaire format and my interpretations of the first set of pilot interviews I decided to conduct a second set of pilots.¹⁰⁹ I employed Monica again for a day to conduct the pilots as she was familiar now with the subject and the case study. We selected the next few names from my sample lists, interviewing three farmers and three employees. After six interviews we agreed that this was a sufficient number to have tested the revised questionnaire, confirmed the principal messages and themes emerging, and provided enough feedback to make final amendments.

109 An example of the second pilot questionnaire can be found in Appendix C.

When finalising the design of the questionnaire for interviewing a larger population I sought further advice from associates at Kadale, based on their extensive knowledge and experience of conduction field research in rural Malawi. This represented a further contribution of the collaboration which Traidcraft facilitated. After

consultation, and as part of my argument for a mixed methods approach, I decided that the most effective format would contain both quantitative and qualitative structured questions.

I decided to add more structure to my questionnaire because with potentially more than one hundred interviews it would be very difficult to analyse a large amount of anecdotal comments. Bryman (2008) discusses this problem of open-ended

questioning that can generate a wide variety of responses that become difficult to analyse, but can be ‘useful for generating fixed-choice format answers’ (2008: 232).

Reflecting on this, I decided it would be easier and more effective to analyse as much of the material as possible by using categories defined by responses to pilot interview questions, therefore, not neglecting the openness of qualitative research in the process of defining grounded themes.

To facilitate that process I developed a questionnaire that contained some questions with ‘tick-box’ options that were determined by the responses from the completed two sets of pilots. I analysed those responses for themes, keywords, and our interpretations that had emerged and categorised them into options for the new questionnaires.¹¹⁰ I, therefore, let the participant material from the pilot interviews drive the final

questionnaire design and the potential answer options. I was confident that this approach of using responses to determine the answer options would not prompt the participants into giving non-spontaneous responses.

Furthermore, this approach maintains the core attributes of qualitative research such as retaining the possibility of capturing nuanced views with open-ended sections throughout the questionnaire. At the same time this approach provided material analysed using both qualitative and quantitative tools to improve credibility and quality. I used SPSS software to collate structured responses to generate descriptive

110 An example of the final questionnaire can be found in Appendix D.

statistics such as frequencies and Microsoft Word to analyse all other unstructured responses collected. Having completed the process of pilot interviewing and questionnaire design, attention turned to conducting a larger number of one-to-one interviews.