Research Quality: Validity, Reliability and Generalizability

Scientific research requires the researcher to demonstrate the validity and reliability of his/her study, making the case that the findings are accurate and credible (Patton, 2002). Further, researchers differentiate between the internal validity and external validity, explaining the latter as “…defining the domain to which the study’s findings can be generalised” (Yin, 2009a, p.40). In case study research, various tactics are used

128

to meet the criteria defined by these tests of quality and the following sections will demonstrate how these tactics were used in this study to meet the demands of the tests of rigour and quality.

5.12.1 Validity

Wasserman and Faust (1994) explain that “...a measure of a concept is valid to the extent that it actually measures what it is intended to measure” (p.57). Research that is deemed to be valid is, by nature, credible as the arguments are logically consistent and the data collected and analysed support the conclusions that have been drawn (Patton, 2002). Yin (2009a) indicates that the quality of the research design in case studies can be judged by three dimensions of validity: construct validity, internal validity and external validity.

5.12.1.1. Construct and Theoretical Validity

Construct validity refers to the process for “…identifying correct operational measures for the concepts being studied” (Yin, 2009a, p.40). In this study of partnerships, attention was paid to construct validity during the literature review and development of the conceptual framework. The constructs were determined from existing research and the relationship between the constructs was developed from the theoretical framework that included the relationship between structure and purpose (Giddens, 1984), the influence of the embeddedness of economic actions in social networks (Granovetter, 1983) and consideration of a range of constructs and concepts from the literature on PPPs, strategic alliances and other forms of partnerships.

Construct validity “…arises when measures of concepts behave as expected in theoretical predictions” (Wasserman and Faust, 1994, p.58). As such, the authors suggest that construct validity of social network measures (and measures attained using other instruments) can be studied by examining how they behave in a range of theoretical propositions. The theoretical propositions that formed the basis for the generation of questions to guide the research were tested in the process of data analysis. Where a concept did not behave as expected, the process yielded a theoretical contribution that met the requirements of originality (incremental or revelatory) and utility (scientific and practical) (Corley and Gioia, 2011).

129

Construct validity is demonstrated in case study research by using multiple sources of evidence, establishing a chain of evidence and having key informants review the case study report (Yin, 2009a). The cross-site analysis was reviewed by one member of staff at Apple who is familiar with all the sites and the findings confirmed (Appendix 10). This independent expert was selected for his knowledge of the region, his familiarity with the business schools, interactions with both academic and private sector actors and institutions, and his expertise in corporate governance.

5.12.1.2 Internal Validity

In this explanatory study of PPPs, internal validity was particularly important (Yin, 2009a; Yin, 2009b). Case study researchers must demonstrate confidence in the analysis of relationships between variables or else “…we are left with interesting stories of what happened, of unknown truth and utility” (Miles and Huberman, 1994, p.11). This research used the analytical techniques of pattern matching and explanation building with the aid of organisational logic and sense-making models to meet the test of internal validity (Yin, 2009a). The logic model used in the analysis is a multi-level model that traces the strategic actions of the schools over time and explains their legitimacy in terms of the public-private relationship.

Two mechanisms within the study greatly facilitated internal validity. Firstly, a mixed- method design utilized data obtained from both qualitative and quantitative analytic processes and, secondly, a multi-level analytical strategy examined variables in the environment, institutional context and actor sub-group level, giving scope to the research. The multi-level analytical strategy is a valid one as the concepts that describe variables in the macro-environment of developing countries are “…global properties (which) are relatively objective, descriptive, and easily observable” and can be generated from the researchers’ analysis of the data (Klein and Kozlowski, 2000, p.215).

5.12.1.3 External Validity and Generalizability of Findings

Several theorists warn that researchers should not commit the perennial error of attempting to generalise their case study findings, explaining that the process of generalisation in case study research is based on analytic generalisation or

130

generalisation to the theory (Yin, 2009a; Perry et al., 1998; Miles and Huberman, 1994). Using the principle of analytic generalisations in case studies, determinations were generalised in so far as there was evidence of “replication logic” predicting similar results or contrasting results for reasons which are anticipated based on the theoretical framework (Yin, 2009a).

5.12.2 Reliability

Reliability is another test of a good quality case study; “A measure of a variable or concept is reliable if repeated measurements give the same estimates of the variable” (Wasserman and Faust, 1994, p.58). The objective in the case study is to demonstrate that if another researcher adopts the same procedures and conducts the same case study they would arrive at the same findings and conclusions (Yin, 2009a). The reliability of this study was facilitated by the use of a case study protocol and by developing the case study database (Yin, 2009a). The protocol carefully described the process used by the researcher and, if followed, should lead another researcher to the same conclusions. The database was also easily examinable and included notes on the observations and reflections of the researcher stored in Nvivo, case study documents, and interview transcripts.