1.4 METHODOLOGY / RESEARCH DESIGN
For data collection, the author kept with the specific objectives outlined under point 1.3. As a working tool, a framework was established with indicators, which were used to verify whether the objectives were reached by CA. For each indicator, corresponding means of verification from primary or secondary data sources were laid down (see Appendix 2).
The following section will give a short overview on the sources of data, methodologies and tools employed to allow for a sound and reliable data collection, analysis and presentation.
1.4.1 Primary research
Primary data collection was a major component of this research. Semi-structured interviews (to a lesser degree unstructured interviews) were conducted with a variety of stakeholders as indicated in the table 1.
Table 1: List of stakeholders interviewed
For every group of stakeholders, adequate interview checklists were developed (see Appendix3, 4). In addition, observations of people, houses and villages were carried out.
Stakeholders No. of
respondents
Type of interview / technique
Beneficiaries 60 Semi-structured
Beneficiaries 52 Unstructured and observations
Members of housing commissions 6 Semi-structured Local government representatives 2 Semi-structured
Local CA staff 5 Semi-structured
International CA staff 12 Semi-structured Representatives of international
NGOs which worked with CA
3 Semi-structured
A major component of primary data collection was the two-week field trip to Kosovo. The sampling methods used to select the respondents, the types of interviews conducted and problems encountered will be discussed below.
Sampling
The entire population (households which received assistance) of the reconstruction programme amounted to approximately 3500. The intended sample was established with 50 households, as this seemed to be a feasible quantity for the time and means available1 to the author.
According to the programme description, the whole population was vulnerable and deserved humanitarian assistance. The population data available did not give any detailed information on the degree of vulnerability of the beneficiaries, but indicated name of head of household, category of the house and number of household members. In the second phase, some beneficiary lists became more detailed, indicating certain vulnerability features, such as women headed household or ethnic minority. As the number of minority households was minimal, they were selected non-randomly.
A two-stage random technique was applied by taking into account geographical and time related features, as these were very specific to the reconstruction programme (Diekmann, 2002). This technique will be described below.
Geographical features
In order to get a sample which represents both locations of intervention equally, the population was divided according to the number of beneficiaries into two sub-groups – municipality Istog and municipality Prizren South/Suhareka (for a better understanding, see map of Kosovo in Appendix 1).
The total number of villages, which benefited from the project amounted to around
1
The author was not working in a team, but conducted all interviews on her own. Therefore the maximum number or interviews per day was established with four, taking into account distances between villages, observations made in between and the time to transfer the information to the computer.
eighty; therefore not every village could be taken into account. Within each municipality, villages were chosen randomly (in proportion to the total number of households in the respective municipality) by selecting every third respectively every fifth village on the list of villages in order to arrive at the number of thirty villages (to conduct one to two interviews in every village). Certain villages were selected non-randomly due to their importance in the project, e.g. for the existence of minority households.
The selected villages then represented sub-groups, from which on the second stage the final sample (respondents) was chosen by selecting every third person on the village list – alternately starting from the first or from the last person on the list.
The following table gives an overview on the number of villages and households (population) per municipality and programme phases and indicates the number of selected villages in proportion to the total number of households.
Table 2: Overview on municipalities, households, villages No. of villages No. of households Municpality 1999 2000/01 1999 2000 2001 Total households % No. of villages Istog 26 43 15002 213 32 1745 50 15 Prizren/Suhareka 31 20 1470 247 0 1717 50 15 TOTAL 57 63 2970 460 32 3462 100 30
The table shows that in Istog 1745 households (50 %) were targeted, whereas in Prizren/Suhareka 1717 households (50 %) benefited from the programme. The number of villages to select in each municipality was proportional to the percentage of households per area (50 % : 50 % = 15 villages : 15 villages).
Time related factors
The fact that the programme consisted of two phases (shelter and reconstruction) was also taken into account when drawing the sample. Within the geographical sub-groups, care was taken that households of both phases were represented proportionally to the budget that was allocated for the year 1999 respectively 2000/2001
2
Table 3: Budget overview
Phase Year Appr. Budget/Eur % Interviews
1 1999 4 600 000 45,5 27
2000 5 000 000 49,5 2
2001 500 0003 5 33
Total 10 100 000 100 604
Table 3 shows that the budget in the first phase was 45,5 % of the total budget, whereas in the second phase it was 54,5 %. Proportionally to the budget, 27 respondents were selected from the first phase and 33 from the second.
Interviews and observations
A total of 112 semi-structured and unstructured interviews as well as observations were carried out. Observations consisted of simply observing the house and the surroundings, sometimes paired with a short unstructured interview, which addressed fundamental questions related to effectiveness and sustainability of the project.
The semi-structured interviews were conducted on the basis of an interview checklist and the duration averaged 40 minutes, depending on the degree of knowledge and memory of the interview partner, his/her ability to give concrete answers to the posed questions and his/her preparedness to participate in the interview. In general, acceptance for the interviews was high; some degree of reluctance was registered in respondents who benefited from the shelter programme (if aid received was comparatively small).
Out of 112 interviews and observations, the 60 semi-structured interviews with the pre-selected respondents were closely analysed. Only for certain criteria, such as effectiveness, targeting or financial sustainability, the unstructured interviews and – if appropriate - the observations were also considered.
3
Only for reconstruction of houses, excluding the social component of the programme
4
The number of interviews was increased by ten due to shorter distances between villages than expected and shorter duration of interviews. The additional respondents were chosen according to the system mentioned above (third person on the list).
Sampling and non-sampling errors
Sampling error compromises the differences between the sample and the population that are due to the particular units that happen to have been selected. By increasing the sample size, the sampling error can be minimized (Erzberger, 1998).
Due to technical and operational circumstances, the sample size of the underlying study was rather low compared to the whole population (60 versus 3500). In order to minimize the sampling error, a two-stage random technique was applied (as discussed above). Furthermore, the additional 62 observations and unstructured interviews that were conducted helped to support the findings and complete the picture constructed through the semi-structured interviews.
Non-sampling error may have occurred due to several reasons. Firstly, although the purpose of the interview was explained to every respondent in great detail, there was still hope among some interview partners to receive additional aid. Therefore questions, especially related to the household’s financial situation, might not always have been answered correctly. Secondly, interviews given by female members of the household might have been less accurate in some cases, as men were more involved in the reconstruction process. Nevertheless interviews were conducted with female household members, as husbands/fathers had already died or were not available at the specific time. Furthermore the women’s perception of the humanitarian intervention and the present situation must not be neglected. Thirdly, some respondents had difficulties to remember certain operational details of the intervention, especially if they benefited only from the shelter programme and got comparatively little support.
1.4.2 Secondary research
The secondary research consisted in collecting and analysing information from existing data. The following main sources were used:
! CA programme documentation, including strategy papers, situation reports, final reports, budgets, correspondence, minutes of meetings, warehouse records, offers and orders, contracts with different actors and distribution lists.
! External sources, such as evaluations conducted by different humanitarian actors5, reports, surveys and statistical data published by UN agencies, ECHO, OSCE, local and national authorities.
5
It was nearly impossible to obtain evaluations from NGOs - for reasons of confidentiality. Evaluations from donors (e.g. ECHO or DIFID) or from UN agencies were more easily accessible.
