Chapter 5: METHODOLOGY AND PROCEDURE
5.7 Study Limitations
The strength and limitations of mixed methods designs have received immense attention from scholars (Creswell 2002; Creswell, Goodchild and Turner 1996; Green and Caracelli 1997; Moghaddam, Walker, and Harre 2003). I have discussed some of the strengths of my design; I will now discuss some of the limitations. First of all, my research design like any mixed methods design requires lengthy time to complete. Considering that this is a dissertation research, this study will suffer from limited time availability. In essence, for a study of this magnitude, more time (much more than my 10 month schedule) would certainly have impacted data collection and improved the results. However, because my design makes effective use of the time available and simplifies the tasks to be accomplished, I am confident that this constraint will not adversely affect the quality of results.
Related to the constraint of time is the limitation imposed by resource availability. More resources will be needed to effectively carry out the various phases of the research than would have been required if the study was designed as a single method study. I expect that this challenge will be mitigated by my success in obtaining collaborative funding from the Social Science Research Council and the United States Institute for Peace. I have submitted proposals to these organizations and expect a favorable outcome. Besides, the cost of conducting research in Nigeria is much lower than what it costs to conduct comparable researches in developed countries. I expect that the outcome of the study will more than compensate for the resources expended on it.
Also, the dependent variable for this study is the respondent’s self-reported predisposition to use violence to protest perceived injustice. Perception is the active, selective and interpretative process of appreciating the world around us. This process, however, is heavily subjective. For
example, one problem with measuring perception is the volatility of what one is measuring. In essence, the perception of respondents can be drastically changed by events occurring in the Niger Delta (such as delay in the payment of monthly stipends, changes in the composition of the supervisory agency, sickness, statements or proclamations from public officials, news items, etc). For example, if we ask respondents whether they believe the amnesty program is successful immediately after the prompt payment of their allowances, we may get a result that will be markedly different than if we asked the same question immediately after a delay in payment of allowances have been announced.
The challenges involved with measuring perception do not mean that perceptions should not be measured. It only means that we need more information and a more dynamic research design to put perception in a more manageable context. My study design does this by effectively grounding the perception metrics. It incorporates multiple metrics that gives each other context. By using multiple methods of data collection and asking different questions related to the violence and the intervention and participants experiences (contextualizing the phenomenon of interest), I am confident that I am able to determine whether the perceptions are accurate or inaccurate.
Finally, like any quantitative survey, this study must grapple with the likelihood of non-response and the overall effect of this problem on my study results. In data collection, there are two types of non-response: item non-response (occurs when certain questions in a survey are not answered) and unit non-response (when a randomly sampled individual refuses to participate or cannot be contacted to participate). The bias occurs when respondents differ fundamentally from non-respondents or when questions differ among the respondent and non-respondent items or units.
The first step for me in dealing with this problem is to create a properly designed survey. My introduction will be personable and professional and the survey itself will feature interesting survey content, short survey length, and clear and concise wording that is attuned to respondent’s burden and likely interest in the survey. Also, I will send advance letters of introduction, provide practical and appealing incentives to respondents ($10 per interview), increase the contact attempts and call backs, and the surveys will be administered face-to-face. I believe these measures will drastically reduce if not totally eliminate the problem of item non-response. Where these still exit, I will discard partial observations and assign weights to each complete observation so that the weighted samples better represents the average characteristics of the population.
For unit non-response, no test or correction for bias can be done without obtaining additional data that includes information about the targeted respondents that failed to respond to the survey. If the number of unit non-response is low, they may have no impact on the study outcome. In that case I may not replace them. But if the number is large enough to bias the study result, I will replace the non-respondents from the list of all registered participants maintained by the Amnesty Committee. The list will be pre-screened to remove names in the first wave of the survey and afterwards, samples will draw from the remaining names to compensate for the unit non-responses. I will also seek to survey the non-respondents to understand their reasons for not responding to the surveys and also to determine whether they differ in certain respects (demography, etc.) from respondents.