Reliability and validity of results

Part of the ethical consideration during a research project is to ensure the trustworthiness of the results. Qualitative data has been highly criticized for a long time as it was not perceived as valid as quantitative data. In the 1950s-1960s, quantitative research was prioritized over qualitative research. According to Charmaz (2015, p.403) “quantitative researchers often viewed the inability of their qualitative colleagues to generate verifiable facts and replicable studies as evidence of lack of rigor”. Guba & Lincoln (1994) insist therefore on the need to establish four points:

● the credibility of the research, that is the confidence in the “truth” of the findings,

● the transferability, or in other words the degree to which the results can be generalized

to a more universal population,

● The dependability. Contrary to the assumption of replicability tied to the traditional view of reliability of data, dependability emphasizes the need to take into consideration the specificities and the continuous change of the context where the study is conducted,

● The confirmability, that is the degree to which the results could be confirmed by others studies.

Lincoln & Guba (1985, p.213) present the credibility “in the eyes of the information sources” of the research as a major trustworthiness criterion. It was especially tested through the four presentations organized in Montserrat and London to the different stakeholders contributing to the post-disaster recovery of Montserrat. Each presentation was the opportunity to get approval, correction, disagreements and specifications. Considering the sensitivity of some topics tackled during the presentations, it was expected to at least observe reactions, if not verbal opposition, in the case of a disagreement. It was also important to have a diversified range of participants to prevent biases. Moreover, the time spent in the field, a total of nine months, gave the opportunity to discuss, informally or formally during an interview, with a large number of stakeholders, and hence has allowed me to regularly specify, correct and complete the previous assumptions. For that it was important to gather a representative sample of the population, with people from different communities and different sectors of the society, to confront the experiences and opinions, while confronting the information with other sources.

Triangulation of data was particularly important as few statistical data exist on the country concerning the topic tackled by the research. Understanding the actual state of the post-disaster situation is essentially based on what people say and think, and is therefore very subject to opinion and prejudices. In addition to the triangulation of sources, triangulation has been conducted by combining research methods. Observation has completed interviews and vice-versa, enabling the clarification of potential misunderstandings. Governmental reports and statistical data have also enabled the cross-checking of the qualitative data.

The analysis of the data also can affect the credibility of the research. Denzin (1978) and Patton (1999) therefore recommend an analyst triangulation and theory triangulation in order to avoid bias of interpretations. The use of multiple framework of analysis and the

Page | 77 understanding of the data in the light of previous research has enabled the confrontation of different interpretations.

The transferability and the dependability of the results are recurrent questions when research is conducted through a specific case study. It matters therefore to identify what findings are specific to the particular case of Montserrat and what can be generalized. This is why a complete understanding of the post-disaster recovery and of the different interactions and influences has been necessary. The small size of Montserrat and of its population enables easier identification of which factors are at stake and influence specific phenomena, and therefore what is particular to Montserrat and what is a more general process. By analysing the case study in the light of more general theories and previous work done in other places, it becomes possible to theorize the study and draw conclusions from the specific case of Montserrat. Chapter 8 comes back in more details on the question of the relevance of this specific case for a broader understanding of the recovery processes.

Finally the confirmability of the study is ensured by using a research design, explained and justified by the specific needs of the study and of the context. Discussion with other researchers and comparison with previous studies enables the verification of the reliability of the methods. Conducting the reflexivity effort presented below is essential to raise awareness and inform about the potential biases in conducting research and what can influence the process and outcomes of the study. By being aware of the pitfalls, it has been possible to combine methods for triangulating data accordingly. It is important to consider these pitfalls if the study is to be reproducible. The methods have then to be adapted to the characteristics of the researcher. The work of data collection has been considered as achieved when the new interviews and observations stopped bringing new information or contradicting the initial conclusions, but seemed instead to confirm the outcomes of the study. However, it may be possible for a researcher very different from me, for instance an older black male researcher, to reach the same findings faster or using another combination of methods as he would face different challenges than the ones I faced as a young white female. This is what the reflexivity effort aims to highlight.