Often researchers are questioned on how their research can be convincing, precise and practical if it is to be repeated by another researcher (Drucker-Godard et al., 2001). Validity and reliability „measures‟ are commonly used to judge the quality of quantitative research. However in a qualitative interpretive research paradigm these questions are debateable and frequently used to critique or dispute the findings of such research work. Qualitative studies involving investigations into the „real-world
setting‟ aim to unfold areas of new phenomenological interest (Patton, 2002). Thus
any kind of research findings produced through understanding or observations of the natural situation are derived from the researcher‟s own perception. Therefore, in comparison with quantitative research which depends on the instruments construction; in qualitative research it is the researcher who is the instrument (Patton, 2002). As a result it is largely the researcher‟s credibility and due diligence in developing the methodology which acts as an indicator or to validate the reliability of the data. The following sections attempt to provide additional information on the measures employed to ensure the validity and reliability of this study.
4.8.1 Validity
The term validity always comes with reliability in both qualitative and quantitative studies. As demonstrated by the numerous definitions of validity, this concept is by no means universal nor static (Golafshani, 2003). From the qualitative research perspective, some perceive that validity is not an applicable term as no single method is universally justifiable (Drucker-Godard et al., 2001). Thus it is recognised that researchers should select an appropriate test to overcome the validation assumptions inherent in this type of research. The extensive reviews by scholars (Miles and Huberman, 1994, Yin, 1994, Eisenhardt, 1989), have led to the development of two distinct validity testing techniques; internal validity and external validity, both of which will be used in this study.
Internal validity consists of internal coherence and pertinence of results produced (Drucker-Godard et al., 2001). It concerns two aspects of qualitative research; causal links to determine whether event x led to event y; and links made between inferences (Yin, 1994). To accomplish this challenge, pilot test interviews were carried out at
initial stage to get further insights into the postulated theory and to simultaneously test the appropriateness of questions for both kinds of respondents. This experiment reduced the probability of making invalid links and sought evidence to disconfirm the assumed link in the research. For example, the identified barriers in UIC research environment are categorised into two major categories and four sub-categories as discussed in Table 3.4. These categories were designed in accordance to the semi- structured interview questions, which were tested in the pilot interview. When no additional data can be retrieved from the pilot groups of respondents, the properties of interview categories are redeveloped. The final scrutinised interview questions were later conducted with another new set of respondents but from the same case grouping.
External validity refers to the degree to which findings can be generalised across settings. The question of generalisability in case studies has always been an area that received considerable criticism (Yin, 1994, Cutler, 2004). The typical question of validity is raised on how a single case or sample can be used to generalise statistical evidence.
Of course the same question can be raised in quantitative research as well; how a scientific generalisation can be based on a single experiment? For example experimental research which is based on multiple set of experiments replicated in different conditions. In short, case studies are similar to experimental studies. It is a form of evaluative research exploring situational conditional differences which may not have a clear single set of outcome (Yin, 1994). It also does not represent samples rather analytical generalisation based on the skills and credibility of the researcher.
The question of external validity in this study is assured by firstly selecting and drawing a robust sample from the population. Since the population are still in its infancy stage as discussed in section 3.4, the drawn samples described in section 4.6.1 are effectively the appropriate informants for the study based on the parameters identified, their background, nature of industry and even nature of projects. Hence by using multiple groups of people of different structural conditions it will maximise the analytical generalisability of this study.
4.8.2 Reliability
In the qualitative paradigm, Lincoln and Guba (1985) described reliability in terms of dependability. In another report, it is referred as the researcher's responsibility for ensuring the research process is logical, traceable and documented (Golafshani, 2003). Reliability addresses how effective the research methods and techniques produce (or can reproduce) data (Cano, 2000). In other words, are the results reproducible by another investigator given the same methodology and an equivalent sample? To ensure reliability is attained, transparency of the research methods used is paramount. This requires a description of the methods used in the research through the use of documented protocols (Yin, 1994). To demonstrate the degree of reliability in this study, some actions have to be implemented such as constructing protocols; getting a dyadic insight into both partner (university and industry) perspectives in natural setting; obtained feedback from respondents and the use of multiple sources of information.
Firstly, to increase the reliability of the study, scholars recommend the development of a set of protocols (Yin, 1994, Cutler, 2004). Thus, in this study, a protocol containing a list of procedures and rules to guide the research process is generated as shown in Figure 4.2. Adhering to the principles recommended, the preparation of protocols help investigators to conceptualise how the findings will be documented. It also provides readers with a structured guide on how the case study has been conducted (Yin, 1994). During the research, another process audit trail is used. This procedure establishes both dependability and confirm-ability of the research process. In all research, an audit trail of the research process should be documented for data analysis and theory generalisation (PTC, 2007). In terms of interview question development, a set of interview guidelines (Gillham, 2005, Kvale, 1996) (see Appendix 4) were followed.
Secondly, to support and strengthen evidence of research, the research gains insights in a dyadic view from both parties (university and their industry partner), thus increasing the reliability of the research. In any circumstance where the sources are inconsistent or conflicting, respondents were contacted to identify any knowledge gaps which required further investigation.
Thirdly, Wahyuni (2003) recommended that one of the most logical sources of corroboration is the respondent. The respondents are after all the key informants in the entire research. Regardless of the reason, the use of these multiple data sources can strengthen the findings while minimising the weakness of a single approach (Wahyuni, 2003). Furthermore by examining the data from a different perspective to corroborate findings increases the reliability of research. Such a data triangulation approach facilitates more reliable interpretation of data (PTC, 2007).