Validity and reliability

A way of strengthening the credibility of a study and its findings is through maximizing its validity. A valid study is one that measures what it intends to measure and one that has properly interpreted its data (Yin, 2015, p. 88). First, the internal validity of this research will be discussed. Secondly, attention is paid to the study’s external validity. Lastly, the reliability of this research is reflected upon.

3.7.1 Internal validity

The question that needs to be addressed in order to determine this research’s internal validity is the extent to which this study measures wat it intends to measure. Firstly, the variables provide valid representations of the concepts they refer to when they are properly operationalized (Toshkov, 2016, p. 118). As the indicators in Table 2 on the previous pages are mostly derived from the examples given in the literature on organizational learning and learning from accident investigation reports, one could assume that the variables, or barriers, are correctly operationalized. Also, the indicators have ‘face validity’, which refers to the ‘common sense’ validity of the indicators used (Toshkov, 2016, p. 118). Secondly, the way in which the data was measured should also be reflected upon in terms of assessing the internal validity. As explained before, it is very well possible that the answers of the interviewees of the Safety Region were influenced by a potential bias in terms of social desirability to protect the image of the Safety Region. In addition, flaws in memory could have influenced the quality of the answers in a negative way. One must bear this limitation in mind.

The principle of triangulation needs to be applied in order to enhance the credibility of the research. The principle pertains to the goal of seeking more than one way of verifying a procedure, piece of data, or finding (Yin, 2015, p. 87). Yin (2015) lists four types of triangulation: data triangulation, investigator triangulation, theory triangulation, and methodological triangulation (p. 87). This research makes use of two of these types.

The first way in which is triangulated is by means of data triangulation, as different employees with different functions within the Safety Region Amsterdam-Amstelland were interviewed, as well as a researcher of the Inspectorate of Justice and Security. The first interview was held with someone who is both the acting secretary of the Safety Board, as well as a crisis management coordinator. The second interviewee is an operations specialist that advises the Safety Board on the organizational process of among other the emergency control room, which is interesting since the 2017 report was specifically on the reachability and continuity of the emergency control room. The third interviewee investigated both power outages and was able to shed light on the Inspectorate’s side of the story. As explained by Thurmond (2001), when collecting data, variances in, among others, persons “add to the study because of the possibility of revealing atypical data or the potential of identifying similar patterns, thus increasing confidence in the findings” (p. 254). In addition, since one of the respondents was able to

Page 38 of 81

provide documents of interest to this study, methodological triangulation is also at issue, since a different method was used for gathering data, which enhances the internal validity of this research.

3.7.2 External validity

The external validity of a research is related to the extent in which the findings of the research can be generalized. Single case studies have often been criticized for offering a poor basis for generalization (Maruster, 2013, p. 377). The external validity of a single case study design is limited, because only one case is under investigation and unless there is an absolute homogeneity of the entire population of cases and deterministic causal links, it cannot be assumed that the findings are generalizable (Toshkov, 2016, p. 304). However, as Blatter and Haverland (2012) explain, at least one of the purposes of studying a single case is to shed light on a larger class of cases. Hence, it makes sense to speak of statistical generalization from a selected case to a population, as one must not forget that in the social- scientific context a single case always remains a case of some broader phenomenon or general reference class (Blatter & Haverland, p. 69; Toshkov, 2016, p. 288). In this research, the Safety Region Amsterdam-Amstelland as an organization is the unit of analysis and case under investigation. Hence, this research is able to shed light on the larger class of cases – the other twenty-four Dutch Safety Regions – in terms of organizational learning from crises and accident investigation reports. In addition, individual case studies can lead to analytic generalization by being embedded in a larger theoretical body of literature. When they test or extend theories, or uncover new causal mechanisms, a particular set of results is generalized to the broader theory that might prove useful for the explanations of other cases (Toshkov, 2016, p. 305; Maruster, 2013, p. 377). All in all, the external validity of this research is to some extent limited. The outcomes of this research are generalizable to the existing theories in the field of organizational learning from crises and accident investigation reports, as well as to other cases that show similar characteristics. These would be the other Dutch Safety Regions. However, it is not possible to generalize the findings to other public organizations in The Netherlands, let alone in the rest of the world. It is important to be aware of this limitation.

3.7.3 Reliability

Reliability is important to test the quality of the research. It implies that if different researchers were to apply the same measurement approach or conduct the research in a different time frame, they would get the same or at least very similar results (Toshkov, 2016, p. 117).

As discussed earlier, it is important to bear in mind that interviews are dependent on the mood of the respondents and the possibility exists of getting socially desirable answers. However, this research’s reliability is strengthened since triangulation is performed by gathering data from both interviews and documents.

Page 39 of 81

In addition, the occurrence of saturation enhances the reliability as well. It implies that all possible opinions on why the Safety Region did not implement the specific recommendations are captured in the administered interviews. Hence, if another researcher were to interview different employees of the Safety Bureau, they would presumably get the same or at least very similar results.

Page 40 of 81