DATA COLLECTION AND ANALYSIS PROCEDURES
8.6. Addressing Quality Issues
A crucial question faced by any research is how good it is. In other words how issues of quality have been addressed? Guba and Lincoln (1994) point out that researches aligning with positivist conventions address quality in light of internal and external validity, objectivity and reliability. However, the same criteria cannot be applied the same way for research studies whereby data is qualitative in nature collected based on people’s perceptions of the relevant issues. Since this study uses the case study methodology Yin (2003) argues that the research should be evaluated against criteria of external validity, internal validity, reliability and construct validity. Further, steps should be taken throughout the process of conducting case study in this regard.
8.6.1. Construct Validity
The issue of construct validity is significantly important in any research to ensure the correct measures are taken regarding concepts under investigation.
Yin (2003) espouses use of multiple sources of evidence and establishing a chain of evidence to address this. Use of multiple sources of data which is also termed as triangulation of evidence is recommended by many others (for example, Eisenhardt, 1989; Patton, 1990; Stake, 1995). They contend that it
results can be enhanced. Referring to a two-case design Marshall and Rossman (1989, pp. 146) point out that using multiple sources of data strengthens ‘the study’s usefulness for other settings.’ This research uses a process which is somewhat circular in nature. Outcomes through multiple sources have been compared and contrasted with literature as well as outcomes from different data collection tools.
8.6.2. Internal Validity
Yin (2003) contends that internal validity is a greater concern for case studies that are explanatory or causal in nature. This research although predominantly exploratory and descriptive in nature, also focuses on determinants of low-tech innovation to explain why or why not innovation in a low-low-tech sector manifests itself. The use of multiple sources of evidence and the reiterative nature of data collection and analysis process (a key characteristic of retroduction) helps address the issue of whether inferences drawn by the researcher are correct or not.
8.6.3. External Validity
Influenced by retroduction the case study approach applied in this research does not attempt to generalize results to the population as is the case with quantitative studies. Yin (1994) argues that the concept of external validity cannot be applied the same way in case study research as is the case in quantitative work. Applying critical realist perspective this study focuses on understanding the mechanisms underlying the phenomenon of LT innovation while also taking influence from previous literature for comparison. Thus the appropriate strategy ‘analytic generalization’ rather than ‘statistical generalization’ (Yin, 2003). The use of multiple case study design and
‘replication logic’ helps increase robustness of research outcomes and strengthens ‘analytic generalization’. Arguing further, Stake (1995) points out that case study research is primarily not a sampling research whereby one case is being studied to understand other cases. Pointing out that lack of sampling in case study research is not a problem it is suggested that;
„…the validity, meaningfulness and insights generated from qualitative inquiry have more to do with the information richness of the cases selected and the observational/analytical capabilities of the researcher than with sample size.‟
(Patton, 1990, pp. 185)
8.6.4. Reliability
The main idea behind reliability is the concept of consistency. That is the ability of data collection and analysis procedures to provide the same answers whenever carried out (Kirk & Miller, 1986) and whether another investigator who follows the same procedures achieves similar outcomes. Guba and Lincoln (1989) support the use of multiple data sources and a trail or sequence of actions taken by the researcher to help an outsider understand how decisions were taken during the course of a study. Yin (2003) suggests the use of case study protocol and case study database to address reliability issues. The protocol developed for this research elaborates the context of the study, the questions it addresses, the conceptual framework used, measures taken during the data collection process to ensure procedures were followed and a set of case and respondent questions used to ensure that the data collection processes remained relevant and on target. Table 8.5 presents the discussion on quality concerns.
Table 8.5: Addressing Quality Issues
Design Quality Criteria Recommended Steps To Meet Criteria Actions Taken Research Phase Construct Validity a. Use multiple sources of evidence
(method triangulation) b. Establish chain of evidence
a. Use of literature review, semi-structured in-depth interviews, structured interviews & questionnaires from multiple stakeholders within marble SSI
b. Results of analysis (Chapters 9, 10, 11) take repeated influence from case study database. The database includes all originally collected data that has been collected in line with case study protocol
Data Collection
Data Collection
Internal Validity a. Explanation building, search evidence for ‘why’ behind relationships
b. Use logic models and displays
a. Testing inferences made and conclusions drawn to ensure important variables have not been ignored b. Models/displays used to establish chain of evidence
Data Analysis
Data Analysis External Validity a. Use replication logic for multiple
case studies
b. Compare outcomes with literature
a. Two case study design (embedded – type 4) used
b. Compare outcomes with literature on LT innovation
Data Collection &
Analysis Data Collection &
Analysis Reliability a. Use case study protocol
b. Develop case study database, systematic approach to data collection and analysis
a. Protocol developed. Presented in this chapter b. Data Collection Plan, memos, codes, tabular
material, interview and questionnaire transcripts
Data Collection Data Collection &
Analysis
8.7. Conclusion
This chapter provided details on data collection and analysis procedures employed in this research. In this regard the case study protocol including the data collection plan applied for the two cases (PeMaS and BuMaS) was provided. This was followed by the justifications for choice of data collection tools and sampling procedures. Different steps of data analysis process were explained including coding, splitting, splicing, memos, within-case analysis, cross-case analysis and others. The chapter concluded with a discussion on measures taken in the research study to address quality concerns including construct validity, internal validity, external validity and reliability. This chapter along with Chapter 7 serve as basis for the three ensuing chapters that present a detailed analysis of data in order to address all research objectives and questions.
Chapter Nine