Research Quality

To ensure the quality of the research findings and conclusions, a number of factors were taken into consideration in the research design. These included measures to ensure the validity, reliability and objectivity of the research findings. The section below describes how the researcher addressed these issues. The researcher also acknowledges the subjectivity likely to result from her own position in the study. This is also addressed.

6.4.1 Validity

There are two types of validity – internal and external validity.

Internal validity of the study refers to the extent to which its design and the data that it yields allows the researcher to draw accurate conclusions (Leedy & Ormrod 2001). Do the findings make sense? Are they credible and authentic (Miles & Huberman 1994)? In other words, do the findings correctly map the phenomenon in question (Silverman 2000)? Qualitative research, with its in- depth access to single cases, has to overcome the temptation of basing conclusions and explanations solely on a few extracts from its field work. The question of representativeness of these extracts threatens the validity of the analysis and conclusions drawn from the study (Leedy & Ormrod 2001). To address this issue Leedy and Ormrod (2001), Robson (1993) and Silverman (2000) suggest use of triangulation of the methods used to collect data. Multiple methods could be used – such as interviews, surveys and secondary data, and within the same method, questions could also be framed in different ways to gather the same information.

The researcher adopted such measures through the triangulation of data collection methods (interviews, questionnaires and secondary data), and the use of quantitative data to complement qualitative data. The exploratory work done with the interviews, and the subsequent coding and analysis provided ground and direction to the quantitative data gathered through the questionnaire. The use of multiple methods also helped to enhance the interpretability of the findings. A qualitative account provided the “thick description” that allows the reader to draw conclusions from the data presented (Leedy & Ormrod 2001) and this was enhanced by supportive quantitative evidence used to buttress and in some cases clarify the account (Robson 1993).

External validity refers to the extent to which the results of a study apply to situations beyond the study itself – that is, the extent to which they can be generalised to other contexts similar to the one in which the study occurred (Silverman 2000).

According to Leedy and Ormrod (2001) the external validity of a research can be enhanced by the use of real-life settings, use of a representative sample and replication to a different context. The researcher adopted such measures through:

(a) Conducting interviews and the mail survey in the real-life settings of the respondents;

(b) Using a representative sample for both the interviews and mail survey, as discussed in section 6.3.4.1.4 and 6.3.4.2.3 on the response rate; and (c) Comparing previous studies with the current study, through a review of

literature, to identify patterns and similarities in settings, findings and conclusions.

6.4.2 Reliability

Reliability is the extent to which findings can be replicated, or reproduced by another inquirer (Silverman 2000). The underlying issue is whether the process

of the study is consistent, reasonably stable over time and across researchers and/or methods (Miles & Huberman 1994). According to Leedy and Ormrod (2001) reliability can be enhanced by using a standardised instrument, and in the case of subjective judgements, specifying the criteria that dictate the kinds of judgement the researcher makes.

The researcher enhanced the reliability of the study through:

(a) The use of a semi-structured interview guide that allowed for some consistency in the way the instrument was administered;

(b) The administration of interviews at approximately the same time each day (usually mid-to-late morning). Interviews were conducted on Tuesdays, Wednesday and Thursdays only. Mondays and Fridays were generally avoided (except at a respondent’s request) as they were perceived to be hectic days in most offices, and adequate attention may not be given to the interview;

(c) A request that the respondents be senior level staff of the organisations. It was hoped that this would increase consistency in the responses based on the assumption that the staff would have a good level of understanding of the issues being investigated;

(d) The administration of the interviews by the same researcher, hence reducing the effect of biases from multiple researchers;

(e) In cases where respondents were expected to make subjective judgements (such as, by using terms like “never”, “sometimes”, “usually” and “always”), an explanation was provided as to what each term represented, to assist the respondents make similar judgements to those of the researcher.

6.4.3 Objectivity

Objectivity refers to the extent to which findings are free from bias (Silverman 2000). Bias is defined by Leedy and Ormrod (2001:221) as any influence, condition or sets of conditions that singly or together distort data. Leedy also acknowledges that the researcher cannot avoid having data contaminated by

some form of bias. However, bias can be reduced and objectivity enhanced. In this context, Robson (1993) defines objectivity as inter-subjective agreement on what multiple observers agree to as a phenomenon.

To enhance objectivity in the study, the researcher used multiple interviews as well as the triangulation method of data collection so as to obtain information from multiple observers with regard to the issues under study. The researcher’s own bias is acknowledged in Chapter One, in sections 1.4 and 1.5 on the assumptions and boundaries of the research to provide the reader with the researcher-position that is useful when assessing the interpretation of the findings and conclusions drawn.

6.4.4. Subjectivity

Subjectivity refers to the extent to which a judgment is based on individual personal impressions, feelings, assumptions, beliefs, values and opinions rather than external facts (Leedy and Ormrod 2001). The researcher approached the study with personal experiences of disagreement and criticism about how donor NGDOs manage development programs in developing countries. In the researcher’s experience recipient organisations in developing countries have little information about the context in which their funding partners operate. They hardly get to ask pertinent questions regarding the development process or some of the actions and decisions taken by their donors. The researcher therefore perceived the relationship between donor and recipient NGDOs as one characterised by tension rather than cooperation. The relationship also appeared to be one of unbalanced power, with donor NGDOs having most power and control over development programs, thus undermining the ideology of partnership with agencies in developing countries – an ideology that they often ascribed to.

With such a perception, the researcher, in using interviews to gather data may have introduced biases in the data. Data may be misinterpreted, either by giving more or less emphasis than intended by the respondent. The tone of voice used, voice inflections and accentuations of particular words could not only

introduce bias but may also have affected how the respondents answered questions (Zikmund 1999; Hughes 2002).

However, the researcher in being aware and sensitive to these possible sources of bias and the personal experiences brought to the study, sought to ensure that such biases were minimised, and the benefits of interviewing (section 6.3.4.1) for this study were maximised.