Social science research can be quantitative or qualitative in style (King, et. al, 1994). Research is designed to make inferences either in a descriptive or explanatory way on the basis of empirical information about the world. Descriptive inference is the process of understanding an intellectual construct on the basis of a set of observations while explanatory or causal inference seeks to identify causal effects from the data observed. Exploratory and descriptive case studies represent appropriate methods to analyze social phenomena.
In deploying case study research, Yin (2003) suggested that unlike a solely qualitative approach, case studies can be based on a mix of qualitative and quantitative evidence. King, Keohane, and Verba (1994) and Brady and Collier (2004) agree that both quantitative and qualitative analysis can be used successfully to achieve similar social scientific ends. Used appropriately, these two approaches complement one another in explaining and interpreting the findings of the study. This research is conducted as an exploratory study with the nested case design and employs both qualitative and quantitative methods of observations, interviews, social network analysis, documentations and structural surveys to identify complementary characteristics rather than advocating a single style of research.
3.5.1 Mixed Methods
3.5.1.1 Qualitative Approach: Observation and Interview
1) Participant observation
I conducted on-site observation in the affected communities and at the emergency
2005, to examine emergency operations and to witness the situation at the peak of the crisis. In participant observation, information from non-structured conversations and documentation were collected and obtained. The purpose is to compare direct observations with other sources of information. Multiple sources could increase the reliability and validity of data and information which the researcher collects for analyses.
2) Interviews
I conducted interviews to get in-depth information of emergency operations and emergency management culture from every level of participating agencies, national, provincial and district. Semi-structured questions in the interview were designed and constructed from preliminary analysis of documentation and participant observation during the first phase of the field research. Detailed information obtained from interviews also supplemented the findings and results derived from other methodologies.
3.5.1.2 Qualitative and Quantitative Approach: Social Network Analysis
I did a content analysis, from a local newspaper (ThaiRath dated December 27, 2004 to January 17, 2005) for a tsunami response, and from the emergency operations reports for the emergency responses in Bangkok area, to extract the information needed for the analysis of social network.
Social network analysis allows measurement of structures and systems quantitatively that would be difficult to describe without relational concepts, and provides understanding of structural properties. Such understanding was gained during a step of content analysis as well as additional information from the observation and interviews. UCINET software which was used to report a measurement of the networks also allowed us to map the relationships of actors in the network into a diagram. This function of UCINET helps visualize how actors interact clearly.
As Wasserman and Faust suggest that the methods of social network analysis provide explicit formal statements and measures of social structural properties that might otherwise be defined only in metaphorical terms, this study presented webs of relationships, closely knit networks of relations, social role, centrality, group cliques and so on are given mathematical definitions by social network analysis. Explicit mathematical statements were also used to compare the characteristics of the networks in this study.
3.5.1.3 Quantitative Approach: Structural Questionnaires
In addition to observation and interviews, this study collected data in a survey of emergency personnel regarding their assessment of work conditions under crises or emergency, inter-agency cooperation, and participation in crisis operations. The survey was intended to serve two purposes; to crosscheck with the reports obtained from observation and interview of similarities and differences, and to identify critical factors for improving emergency operations and building a strong emergency response network.
A stratified representative sample was used to collect data from emergency personnel at national, provincial and local levels of emergency operations and cover three groups of management levels which are top, middle and operational. How to distribute and conduct the survey and a description of the sampling design are presented in data collection.
3.5.2 Threats to Validity and Reliability
This study reviewed the criteria for internal validity, external validity, and reliability of the findings and analyses to determine whether there were inconsistencies that affected the generalization of findings across time and space.
A threat to internal validity was rose to question the effect of intergovernmental coordination, information sharing and communication management, and socio-technical components in emergency management. The exploration and assessment of this study were built upon the studies and practices of disaster and emergency management in complexity. Participant observation and interviews allowed this study to document and obtain information from the informants directly how they viewed the effects of factors proposed by the study on emergency responses to avoid such threat.
Threats to external validity arose differently since this study covered every emergency agency and organization with in the nation, Thailand. Instead of questioning whether research findings could be generalized with other actors or jurisdictions, the question was asked if the findings could be used by other developing countries and other types of emergency such as terrorism. This study emphasized the significance of culture and emergency structure as factors affecting to emergency response. Further study needs to take into consideration the similarities and differences in cases’ characteristics if findings are analyzed in different settings. The other perception of external validity was the extent to which this study was drawn on the theoretical framework of “Shared Risk”, Comfort 1999. Her socio-technical components were also used to apply and examine other emergency responses in other countries across the world. This study, although quantifying socio-technical components, shared the same theoretical fundamental as in Comfort’s model. This study validated this concept of strong relationship between socio-technical components and emergency response in the next chapters.
Construct validity was taken into account of the study emphasizing empirical studies and practices of sources and theoretical frameworks upon which the operationalization of inferences is built. This study carefully modified socio-technical components from previous studies and
practices to fit with specific characteristics of the Thai environments using preliminary field research and analyses of experts. The proposed conceptual model of critical roles of intergovernmental coordination, information sharing and communication management were evidenced by the event of ineffective tsunami response. A preliminary analysis of Bangkok emergency response from non-structured interview also reflected an agreement of the acceptance of critical roles of those factors in emergency responses.
Reliability has to do with the quality of measurement. In its everyday sense, reliability is the consistency or repeatability of the measure. This study paid attention to two types of reliabilities; 1) Inter-rater or inter-observer reliability, and 2) Internal consistency reliability. For the inter-rater reliability, this study was able to ask specific questions twice in a different way to determine the accuracy of the responses being provided. For the internal consistency reliability, this study performed a pre-analysis data screening before running factor analysis and multivariate analysis to make sure that all assumptions and restrictions are met. Tests of missing values, outliers, normality and linearity were performed. In addition, two different persons were performing manual verifications. However, reliability test can be performed mathematically by computing Cronbach’s alpha value or calculating the estimation of a ratio between variances of true score and measured score. The closer to “1” the ratio is, the higher reliability the measurement has. Each of the reliability estimators gives a different value for reliability. In general, the test-retest and inter-rater reliability estimates will be lower in value than the parallel-forms and internal consistency estimates because they involve measuring at different times or with different raters.
We often think of reliability and validity as separate ideas but, in fact, they are related to each other. In many cases, the study is consistently and systematically measuring the wrong
value for all respondents, or the study seldom hits the target but gets the right answer on the average. Statistical validity and reliability are a strong part of this research due to the coverage of cases and large sample size. Cases used in this study cover emergency response of national, provincial and district levels managed by every emergency personnel and organization in the nation. Sample size is large of total number of 424 informants (N=424) which include emergency personnel around the country, 75 provinces and a capital city, Bangkok province.
Multiple methods used in this study also complement each other. Generalization of quantitative analyses of statistical findings is also well explained in detail and depth of understanding by the qualitative analyses of interviews and documents review. Social network analysis as a qualitative-quantitative tool also allows this study to explain emergency response network in statistic interpretation and narrative assessment. Nested design helps relate the cases this research study to identify factors affected to both individual and networking emergency responses.