The research process was accompanied by a multitude of methodological challenges. The specific research design, the methods applied during field research and the mixed-method paradigm entailed not only advantages but also methodological problems. The researcher had been aware of these challenges and addressed them to the best of her knowledge and beliefs. Nevertheless, the data are still bound to uncertainties and are not always representative. In order to be able to draw solid and reliable conclusions from the empirical results of this study, it is important to address these challenges at this point and keep them in mind while reading the rest of the thesis.
Firstly, the mixed-method case study design not only drew on the advantages of qualitative and quantitative research but was also accompanied by specific challenges. Having chosen to apply a multitude of methods in a case study approach under given time and budget constraints implied restricted representativeness. The research was conducted in only one province of one agro- ecological zone. For that reason, representative conclusions cannot be drawn for the whole Mekong Delta – a region of notable social, economic and natural diversity (see chapter 3). Moreover, the sample size cannot meet representativity requests with regard to all research objects for the chosen research area. Qualitative research does not typically underlie the requirement of being representative, but quantitative research does. This problem therefore arose most notably in the context of the structured household survey. The sample size was initially calculated to fulfil at least the minimum prerequisites for conducting most statistical analyses at a representative level (for the research area). Given the length of a questionnaire which aims to evaluate a large range of strategies, a trade-off emerged between being comprehensive, reliable and valid or being representative and able to provide a solid basis for certain statistical analyses. The researcher struck a balance by bisecting the survey population. Around 100 interviews were targeted for a long questionnaire-based interview of one to two hours and 200 interviews for a shorter interview of half an hour to one hour. The long and the short questionnaires were identical except for the fact that the long questionnaire included an evaluation part at the end of each appraised strategy section (see sample questionnaires in Annex 12.5.4). In this way, a sound basis of comparison with a sample size of 312 was achieved for assessing vulnerability patterns, risk perception, as well as governmental and household-level strategy options. The evaluation of strategies was based on a smaller sample size but did still allow for descriptive analyses and comparisons (see Annex 12.3 for more details).
From the perspective of an explorative research design (see previous section 4.4), it would have been pertinent to conduct more in-depth interviews prior to the standardised survey. This would have provided a more profound basis for ensuring sufficient comprehensiveness in the extent of questions and adequacy with regard to the formulation and answering categories. Nevertheless, the required change in research sites and the restrictive research permit requirements prohibited generating more qualitative data in the forefront (see section 4.6). Within the realms of the rigid system of governmental research restrictions, it was, however, possible to extend the scope of pre- tests before the survey. Two rounds of pre-tests were conducted, one which included fewer but more in-depth interviews conducted by the researcher; and a second round where enumerators undertook a larger range of standardised interviews according to the questionnaire. In that way, it was possible
42
to improve the questionnaire design and compensate for some of the drawbacks resulting from a lack of qualitative research before the survey (see Annex 12.3 for more details).
A drawback which arose from aiming at a larger representativeness was the fact that not all interviews could be conducted by the PhD researcher. Most of the survey interviews and many of the production- and migration-centred interviews were conducted by enumerators. This entailed a lack of reliability especially from a qualitative stand. The notes taken throughout the interview were, for instance, not as detailed and it was difficult to judge the atmosphere during the interview. In order to ensure as much reliability as possible, experienced enumerators43 were chosen and trained extensively. For the survey, one day of training was provided before the pre-tests and two days after the pre-test. Moreover, feedback rounds with the enumerators were undertaken in the lunch breaks and after each day of interviewing. On these occasions, everyone described his/her experiences with the interviewees and ambiguities were jointly discussed. In addition, two enumerator group leaders were appointed for the survey who crosschecked each questionnaire in order to counteract data gaps and clarify doubtful and indistinct points shortly after the interview (see Annex 12.3 for more details).
Moreover, the complexity of some research aspects reoccurred in the responses of several interviewees. This was most notable in the abstract notion of quality criteria. It was, for instance, difficult to appraise general quality criteria in the group discussions. Both government authorities and particularly the households had difficulties in defining general characteristics which make a strategy a “good” or a “bad” one. Illustrating the nature of criteria using the example of specific strategies would have directed the selection too much towards an evaluation of only this strategy and providing no example made it difficult to imagine what criteria were. Therefore, the researcher chose to provide examples from a list of criteria derived from a review of the literature and ensured that if examples for strategies were used, they were the same across all group discussions. Moreover, it was important to discuss potential explanations for criteria extensively with the translator and with other Vietnamese researchers to arrive at formulations as understandable, comparable and precise as possible (see Annex 12.3 for more details). In the survey, the researcher chose to ask for a ranking of specific advantages and disadvantages of each household strategy and a likert scale question format for the evaluation of government strategies. These questions were often difficult to answer for the survey interviewees. To ensure comparability and comprehensibility, the survey assistants were trained to use the same examples and formulations for each strategy which were thoroughly selected beforehand (see Annex 12.3 for more details). Also the institutional setting was a complex topic to assess. The sampling of interviewees was therefore of central importance. In order to select appropriate interviewees, the enumerators were asked to register the survey interviewees who were the most open, interested and knowledgeable with regard to the institutional setting. From this list of potential interviewees, one candidate from each commune and production type was chosen. These people were not interviewed in a one-to-one interview format but in a participatory group discussion to which the interviewee invited other people with whom he/she felt comfortable. The relational PRA method of Venn Diagrams appeared to be the most appropriate way of revealing implicit knowledge with regard to the complex network of institutions, responsibilities and interconnectivities (see Annex 12.3 for more details).
Furthermore, the reliability of interview documentation was restricted because neither interviews nor discussions were audio-recorded. Therefore, no word-by-word transcriptions were available for the content analyses. For that reason, not all information given in the interviews could be registered and several content analysis tools such as dictionary-based approaches were only possible to a limited degree. Reasons for restraining from audio-recording were most notably related to the fact that a recording deterred interviewees, particularly authorities, from providing more in-depth and critical information. Instead of recording, the researcher took detailed notes. In order to gain reliable information, the translators were instructed to translate as precisely as possible during the interview.
Moreover, each interview was discussed with the translator right afterwards to resolve unclear points, complement and crosscheck the notes, and jointly interpret the interview atmosphere and the openness of the interviewee.
Also the process of logging in questionnaire data was a source of restricted data reliability. It hinges on the quality of data entry. The assistants were therefore thoroughly selected and trained. The data entry was undertaken by the two enumerator group leaders who not only knew the survey design well but also cross-checked each of the questionnaires during the survey. To improve the quality of their work, a data entry excel sheet was created and translated in coordination with an experienced Vietnamese researcher from the Mekong Delta Development Institute. It was neatly ordered and numbered based on the questionnaire design, the questions and answer options were rationally and simply coded, missing values and the level of measurement was defined, and automatic error reports were programmed so that entry mistakes could be reduced. The data entry assistants then received a one day-training and conducted the first data entries together with the researcher. Throughout the data entry period, spot checks on a random basis and feedback rounds with the data entry assistants were conducted (see Annex 12.3 for more details).