METHODOLOGY: ASSESSING LESSON-LEARNING
CAPABILITIES OF POST-DISASTER INQUIRIES
After illustrating existing issues and gaps within the literature on disasters, governmental responses to them, royal commissions, and post-disaster inquiries, it is important to outline how this thesis intends to adequately address these and its research aims. This thesis employs a comparative case study method that allows for “valid generalisations provided that there is a theoretical statement against which to compare the case studies” (Rhodes 1995: 56). The method is adopted because it is the most effective and appropriate method for evaluating and addressing concerns with disaster responses, including post- disaster inquiries that were outlined in the literature review (Chapter 2). This chapter outlines the rationale for the adoption of the comparative case study approach and differentiates between the chosen methods, methodology, and research design.
The comparative study undertaken in this thesis is important because of the continued and increasing frequency, threat, and impact caused by a range of disasters in Australia, but also due to a need to learn and identify lessons that ensure their impacts are mitigated or prevented in future. The need for this was identified in the literature review of this thesis, where an incremental approach to policy development, a combination of the ‘top-down’ and bottom-up approach to policy implementation, and Downs’ issue- attention cycle were discussed as relevant to the undertaking of post-disaster inquiries. In the selection of these cases, the desire to discuss decision-making in relation to these policy concepts through the establishment, investigation, and post-investigation stages of post-disaster inquiries are considered. The aim of the comparative analysis conducted by this thesis is to highlight how a successful post-disaster inquiry can occur, so that it informs lessons that shape the direction of disaster preparedness and responses in a way that the public accept and recognise.
This chapter begins by discussing the methodological ideas that underpin this study. Included in this is a discussion of how ontology and epistemology, as well as qualitative and quantitative research ideas underpin choices in the design of this thesis. Following this, its research design is outlined, where the discussion focuses on why the comparative case study method is adopted and how this is utilised. Lastly, this chapter outlines the methods utilised to conduct this research and the rationale for these. In support of its chosen approach this thesis draws on contributions from scholars, including Arend Lijphart (1971), Harry Eckstein (1975), Alexander L. George (1979), Robert Yin (2003) and John Gerring (2004). It will also cite other scholars who focus on research methods, specifically those from within the political science discipline.
57
Discussion of methodology
It is important to understand the methodology that underpins the research’s design, especially as it helps set up this thesis and address its key aims. Choices between different methodological tools affect the direction and outcomes of this thesis and it is important that the right balance is adopted to address its aims. These choices are discussed further throughout this section.
Ontology and Epistemology
The capacity for inquiry and ability to ask questions is vital to the development of research. It allows for the interrogation and challenge of differences between, assumptions made, and the knowledge produced by specific theoretical and analytical traditions. Ontology and epistemology are two different approaches for viewing research philosophy. These concern our views of reality and theories of knowledge while also underpinning theoretical perspectives and methodologies. Ontological questions concern the nature of social and political reality, while epistemological questions relate to knowledge and the justification of this. Together, these questions are the foundation from which opposing research perspectives are built.
David Marsh and Gerry Stoker (1995; 2002; 2010), as well as Colin Hay (2002) have contributed to developing understandings of ontology and epistemology, particularly in political science. Hay (2006: 8) observes that ontology precedes epistemology and argues that “we cannot know what we are capable of knowing (epistemology) until we have settled on (a set of assumptions about) the nature of the context in which that knowledge must be acquired (ontology)”. Similarly, Blaikie (1993: 6) suggest that political ontology “refers to the claims or assumptions that an approach to social [or, by extension, political] enquiry makes about the nature of social [or political] reality—claims about what exists, what it looks like, what units make it up and how these units interact with one another.” The questions this raises, include: “[w]hat is the nature of the social and political reality to be investigated? Alternatively, what exists that we might acquire knowledge of?” (Hay 2011: 464). These questions assist in the development of the research’s ontological approach, as it helps understand the political reality in which post-disaster inquiries exist, and thus assists with the development of the comparative case study method.
While ontology asks “what exists to be known?”, epistemology asks “what are the conditions of acquiring knowledge of that which exists?”. Epistemology is defined as the science or philosophy of knowledge (Hay 2011: 465-466) and discusses “the claims or assumptions about the ways in which it is possible to gain knowledge or reality” (Blaikie 1993: 6-7). Thus, it refers to how defence of a preference between competing political explanations is settled. This allows for decisions about whether
58 legitimate generalisations between cases can be made. When adopting a qualitative approach throughout a comparative case study method, it is important that accurate generalisations are made about the cases. This is especially important in this thesis, specifically as it examines the lesson-learning capabilities of post-disaster inquiries.
Accordingly, it is important that ontology is understood to be associated with a central question of whether social entities need to be perceived as objective or subjective. The study of post-disaster inquiries includes both objective and subjective data. Interpreting this in its analysis requires an understanding of the political reality that exists around this research.
Qualitative and quantitative research, and the use of secondary sources
Often, research will independently adopt either qualitative and quantitative methods, or a combination of both, however, choices between the two are dependent on the research being undertaken. Qualitative research is an exploratory tool that is primarily used to gain an understanding of underlying reasons, opinions, and motivations of a problem associated with an aspect of social life (Eckstein 1975; Read and Marsh 2002: 232). Eckstein (1975) observes that the approach provides an insight into the specific problem. This thesis adopts a qualitative approach as it seeks to understand the use and utility of post- disaster inquiries. Observations of previous post-disaster inquiries demonstrate the underlying values of individuals and communities to social understandings of a problem (Pierce 2008: 45). Marsh and Stoker (2010: 257) view complete objectivity as impossible to achieve. This thesis seeks to provide an empathetic understanding of post-disaster inquiries and in doing so illustrates the varied and objective role of various stakeholders.
The comparative case study approach of this thesis is undertaken with a small sample size, which is associated with qualitative approaches and provides a deeper analysis of attitudes, feelings, and behaviours of agents and institutions in a disaster’s management. However, this thesis is limited by its reliance on secondary sources, such as: relevant academic journals and studies, media reports, and relevant post-disaster inquiry reports. These sources are valuable to this thesis because they offer existing analysis and an understanding of key issues discussed. However, the absence of interviews and primary sources beyond the use of post-disaster royal commission reports is a limitation of this study, because the researcher did not participate in the data collection process and is unaware of how this was conducted. Therefore, it is impossible to make judgements on how well the processes behind the collection of data were conducted and whether these were affected by any problems. In this study, the examination of existing data, in relation to its key research question assists it to overcome these limitations.
59 Secondary source analysis is an empirical exercise and is argued to apply the same basic research principles as studies utilising primary data (Johnston 2014: 619). Similar to other research methods it follows a systematic set of steps to reach its conclusions. Johnston (2014: 619) asserts that secondary data analysis is a viable method for research, specifically when a systematic process is adopted.
Qualitative approaches are also able to expand upon topics that lead to examinations of areas not traditionally considered through discussions of why people act in certain ways and their feelings about these actions. This view of qualitative research is used to frame this examination of post-disaster inquiries, the decisions within their processes, as well as the impacts of these. The use of qualitative methods is therefore preferred over quantitative methods, because it generates data regarding the underlying reasons and opinions of an aspect of social life that is sought by this thesis. Contrary to this, the quantitative approach identifies statistics that are utilised by researchers to quantify attitudes, opinions, behaviours, and other defined variables (Creswell 2013). As Bryman (1998: 94) asserts, while “quantitative research is hard and reliable […] qualitative research is deep and rich.” Competing academics have historically utilised quantitative or qualitative approaches (Pierce 2008: 41). Due to recent changes, Read and Marsh (2002: 231) observe that:
[m]ost empirical researchers acknowledge that both qualitative and quantitative methods have a role to play in social science research and that, often, these methods can be combined to advantage.
Despite the possibility of using both methods, a qualitative approach is adopted in this thesis. This is due to it considering research as a process that helps develop understandings of contextual meanings and socially constructed realities (Moses and Knusten 2007: 191-195). Qualitative data provides a rich and detailed picture about why people act in a certain way, with a focus centring on their feelings about these actions. This is important as this thesis seeks to observe how the actions of decision makers and various other actors throughout a post-disaster inquiry impact the success of its investigation and outcomes. Along with this, it seeks to understand why these decisions are made and whether alternative methods of policy evaluation are available and appropriate.
A further rationale for the adoption of a qualitative approach is the chosen comparative case study method. This approach aids not just the study’s exploratory or descriptive purpose, but provides a comprehensive process of constructing its research design. George (1979) and Yin (2003) prescribe that establishing a comparative case study approach encompasses research design, data collection methods, data analysis, and theory building. Setting up comparative case study research involves a series of tasks. These include reviewing the existing literature, conceptualising the research problem, specifying the outcome requiring explanation, and establishing the control variables for the case studies, as well as the
60 variables that comprise the theoretical propositions and framework of this thesis, case study selection and case study analysis (George 1979: 54-55).
The comparative case study method: research design
Gabriel Almond (1990: 36) recognises that political theories and phenomena differ from reality and that political scientists are seen to investigate issues considered ‘soft’ and less empirically precise, rather than the ‘hard’ and easily calculated evidence used in other disciplines. Despite the acceptance of this position, political theories should not be viewed as any less ‘real’ than others, especially when previous evidence and analysis illustrates the important new and alternative knowledge they convey (Almond 1990: 36). However, it does dictate a strong difference between the measurability of strict scientific studies over that of social science alternatives.
Much of the scholarly contributions relating to public policy are multi-disciplinary. As such, policy studies involve significant input from other scholarly disciplines, especially economics, sociology, and law (Fischer 2003), but must also include contributions from disciplines with relevant expertise. Analysts who possess a broad range of social scientific understanding, quantitative expertise, and communication skills are well placed to contribute to the study of public policy (Fischer 2003). Fischer (2003) also observes that public issues are not neatly contained within traditional disciplinary boundaries. This thesis includes input from various scholarly backgrounds, to ensure a broad range of knowledge and expertise is exhibited of the available literature.
Despite alternative methods existing, a rich history of case study research exists in political science which has transferred into areas of public administration and public policy studies (Eckstein 2000). A vibrant and contemporary methodological discussion within comparative studies of political systems also exists (Eckstein 2000). This chapter proceeds by outlining the comparative case study approach and why it is appropriate for this thesis.
Comparative case studies
Gerring (2004: 341) describes a case study as “an intensive study of a single unit for the purpose of understanding a larger class of (similar) units.” He discusses the “disjuncture that exists between the case study’s acknowledged contributions to political science and its maligned status within the discipline […]” (Gerring 2004: 341). It is the most widely adopted method in political science (Bennett and George 2005: 61). Despite this, political science has no set definition or method of application for a case study. Instead, a series of definitions contribute to a growth in its understanding, but fail to fully inform political scientists what a case study is and how it is best utilised.
61 A vibrant and contemporary methodological discussion within comparative studies of political systems also exists (Eckstein 2000). Yin (2003: 13-14) asserts that any definition of a case study should involve characteristics that stipulate its data collection and analysis requirements. He defines case study research as “an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident.” (Yin 2003: 13). Specifically, Yin (2003: 13-14) observes that:
[t]he case study inquiry copes with the technically distinctive situation in which there will be many more variables of interest than data points; …relies on multiple sources of evidence, with data needing to converge in a triangulating fashion; and …benefits from the prior development of theoretical propositions to guide data collection and analysis.
This research adopts a comparative case study approach as it looks at post-disaster inquiries, which involve many variables. The adoption of a comparative case study approach is appropriate because conditions, issues, or ideologies are shared across post-disaster inquiries established in various similar jurisdictions. Their selection is dependent on the number of variables that are to be examined, the availability, and access to related, comparable data, data format, both numeric or non-numeric, and the adopted means of analysis (Pierce 2008). Engels’ (1845/1993; cited in Pierce 2008: 56) illustrates five main reasons for undertaking comparative political research:
1. to provide rich, contextual descriptions which identify clearly the observed similarities and differences between cases and places;
2. to identify and develop systems of classification and typologies which generate data sets appropriate to the research question;
3. to distinguish independent variables (causes) from other variables; 4. to test hypotheses and thus, the validity of explanatory theories; and 5. to develop predictive capacity where, for example, there is evidence of
stages of development.
Despite these desired outcomes, a series of common critiques and misunderstandings are associated with the case study approach. Flyvbjerg (2006: 219-45) lists five common misunderstandings associated with case study research:
• theoretical knowledge is more valuable than practical knowledge;
• one cannot generalise from a single case, therefore the single case study cannot contribute to scientific development;
• the case study is most useful for generating hypotheses, while other methods are more suitable for hypotheses testing and theory building;
• the case study contains a bias towards verification; and • it is often difficult to summarise specific case studies.
Flyvbjerg (2006: 219-20) examines and evaluates the pitfalls of these misunderstandings so to limit any oversimplification or inadequacies and to make the comparative case study approach a stronger and more widely accepted research method. He also notes that “good social science is problem-driven and
62 not methodology-driven” in the sense that it employs those methods, which for any given problem best help address its research question (Flyvbjerg 2006: 235-36).
This research adopts a comparative case study approach to address the utility of post-disaster inquiries, especially their ability to inform lesson-learning that improves disaster preparedness and responses. The approach is favoured due to similarities in the adoption of royal commissions in response to catastrophic disasters, but also a desire to evaluate the effectiveness of current disaster responses methods.
Case study research is utilised in the study of public policy as an analytical tool that illustrates the actions of various policy actors who attempt to influence the policymaking process (Eckstein 1975). Public inquiries, especially royal commissions are tools utilised by executive government to shape and gather information in the policymaking process. Through case study research, any detail of a decision or set of decisions can be analysed to identify constraints on, approaches to, and techniques for decision- making processes (Eckstein 1975). When ideological perspectives underpin the case study design, the legitimacy, or appropriateness of theoretical concepts are questioned, despite this, its presence is to be expected, especially in the study of politics which is inherently ideological (Eckstein 1975). The use of case studies allows for further learning about how public policy is designed, made, and implemented.
The comparative case study approach acts to “maximise the variance of the independent variables and minimise the variance of the control variable” (Lijphart 1971: 164). The selection of cases for this thesis is based on key similarities, namely the establishment of royal commissions to investigate a disaster. A comparative case study approach is adopted to evaluate the role that post-disaster inquiries perform in the design and implementation of disaster policy. Post-disaster royal commissions emerged from the Westminster system of government and generally share the common characteristics of being established following a disaster. As such, a ‘most similar’ approach to case study identification is adopted. It utilises Lijphart’s (1971) understanding of how to select cases, which helps ensure that selected cases are suitable for comparative analysis to achieve the greatest insight.
The ‘most similar’ approach allows for the key differences of each case to be isolated, something which leads to a more focused analysis. In this regard, the chosen method of this thesis aligns with the ‘most similar’ approach, because all three of its cases are post-disaster royal commissions that identify lessons aimed at improving preparedness and responses to future disasters. Three main variables exist in its case selection, namely the disaster type being evaluated; the location, and the impact of the disaster and inquiry; and the responsible government authority for the post-disaster royal commission’s establishment. The latter is controlled through the selection of cases originating from jurisdictions where similar Westminster traditions of government are present. This thesis identifies how decision- making regarding the operation of these selected post-disaster royal commissions differs, due to
63 national, and state-based cultural differences in how the approach proceeds. The effect of differences