In this chapter the constraints and obstacles in establishing validity and reliability in this research will be discussed.
Reliability means that in repeated research in scale and time, the same results will come forward. The methods used to measure the data are thus consistent (Kumar, 2011).
Measuring validity means looking at what is researched and how it reflects to reality. Measurements can be consistent across time and scale (reliable), but if measurements are consistently wrong the research does not accurately reflect reality: there is no validity (Kumar, 2011).
Validity
In this research vignettes and interviews are used. The first obstacle encountered in using vignettes is the fact that this research is trying to measure what actions the population would undertake in a crisis situation and why they make those decisions, but the research can only focus on the intentions of people who have probably never been in such a situation. Some researchers have found that this gap between intention and actual action is very small, while others report a clear gap (Barter, 1999). We do not know how severe the gap will be in this research, and this can be a problem for
establishing validity. This is an inherent problem of the crisis management field and the research method. Several things can be done to constrain this lack of validity. Hughes (1998) in his research on vignettes and its application on drug users found that vignettes mimicking reality as close as possible worked better. It also helped if a situation in a vignette is followed up in several steps, as if it was a small story. A maximum of 5 times is recommended to prevent confusion in the
respondents. In this research, the vignettes have a maximum of three steps. The small story increased the emotional connection and empathy with the protagonist and the situation in general, which led to answers which better portrayed reality (Hughes, 1998).
The internal validity is increased due to the semi-structured interview style, which matches the goal of researching the perception of the respondents. Respondents can talk freely about their
experiences and expectations, and the semi-structured style also allows for in-depth questions into topics which are brought up by the respondents, this can lead to a better result of the research. External validity will be difficult, as this research has a (self-)selection bias, and the amount of respondents (24) is not high enough to be conclusive. Furthermore their geographical locations are confined to two cities, Venlo and Utrecht. The data gathering methods used the personal network of the interviewer to find initial respondents, and a snowball method to find more respondents. This makes it difficult to generalize the results of this research to all adolescents in the Netherlands. What it can do is offer avenues for further research and offer an insight in how some adolescents perceive crisis situations.
Socially desirability bias
A general problem in social science is the bias in giving socially desirable or acceptable answers, respondents do not answer truthfully, but modify their answers to be socially acceptable. Anonymity lessens the impact of this bias. The participation and answers of the respondents are anonymous, which increases the validity. However, the interviews themselves are in-person so it is still possible this bias has an effect. Vignettes also make it possible to mitigate a part of this problem. In the vignettes we follow the actions of another person. We can then compare what this protagonist of the story does with the actions of the person being interviewed and his expectations of what the
discussed, which shows more of the initial thought process than just the (socially desirable) answer (Finch, 1987). Secondly, the increased empathy with the protagonist also makes socially acceptable answers less likely (Hughes, 1998); this increases the validity.
Reliability
This qualitative research becomes repeatable by other researchers through the use of the semi- structured interviews in which all topics are clearly indicated. Although it will be difficult to replicate the interviews exactly, as not all questions are written down in advance in a semi- structured interview. But the topics and manner of interviewing are clearly described and can be replicated, increasing the reliability. The vignettes are described likewise.
The physical setting of the interview is always one which is familiar to the respondent, for instance a classroom and the respondents are also given the same explanation of what constitutes a crisis so that they have the same basic knowledge, which both increase reliability (Kumar, 2011).
In order to increase the reliability, the concepts of other qualitative and quantitative research are used. The theories used are clearly noted in the concept operationalization and can be read and used by other researchers as well.
The final codebook is made available and can be used, this is added in attachment 4. The Atlas.Ti software will be used to track and code all data in one database.
6. Results