Validity and Reliability

Validity is the degree to which research findings precisely represent what is really happening in a situation. According to Collis and Hussey (2009), validity in positivist research is very low; phenomenological research attempts to capture the essence of phenomena, extracting data that are rich in terms of explanation and analysis, so that validity is high under such a philosophy. It does not attempt, by any means, to pre-determine the data or structure of the

67

research. In qualitative research, validity attracts more support than reliability. In positivist research, the researcher needs to design the measurement parameter, the means of measuring and also the framework for data collection. Therefore, validity will be affected by how well the framework is designed.

Reliability relates to the findings of the research, and is one aspect of the credibility of the findings. If the research is repeated, then the same results should be obtained. Reliability in positivist studies is usually very high, while in phenomenological studies it is very low, and researchers should follow a number of procedures to ensure reliability (Collis and Hussey, 2009). The problem with this is that it is not possible to be sure that the individual, and other factors, have not changed between the two occasions. Ideally, a test for validity and reliability should be made at the pilot stage of an investigation, before the main phase of data collection (Easterby-Smith et al., 2008).

Cavana et al. (2001) concluded that validity is concerned with whether the researcher measures the right concept, and reliability is concerned with stability and consistency in the measurements.

Yin (2009) acknowledged that there are four tests used to establish the quality of empirical research:

 Construct validity: establishing the correct operational measures for the concepts being studied. In order to strengthen construct validity, three tactics are available:

 Use multiple sources of evidence: this is relevant and very important during data collection. In order to increase construct validity, the researcher makes use of many sources of evidence, such as semi-structured, face-to-face interviews as the main source of data, in addition to other sources, such as documentation, archival records and direct observation.

 Create or establish a chain of evidence: this is also very important during data collection.

 Have the draft case study report reviewed by key participants: this relates to a review and validation of the draft interview by the respondents to reconfirm their expression during the interview. This will confirm their comprehension of the interview questions as well.

68

 Internal validity (for explanatory or causal studies only, and not for descriptive or exploratory studies): establishing causal relationships, whereby certain conditions are shown to lead to other conditions, as distinct from spurious relationships. Riege (2003) stated that the emphasis, in constructing an internally valid research process in case study research, is on establishing phenomena in a credible way. He added that the researcher does not only highlight major patterns of similarities and differences between respondents’ experiences or beliefs, but also tries to identify the components that are significant for the patterns observed, and the mechanisms that produced them. Yin (2009) suggested four tactics for achieving internal validity (see Table 8). In this research, internal validity was achieved by using explanation building methods to analyse the data gathered. The use of the theoretical framework developed also helped to increase the internal validity.

 External validity: establishing the domain to which a study’s findings can be

generalised (Yin, 2009). Riege (2003) argued that case studies rely on analytical generalisation as a means of achieving external validity, whereby specific findings are generalised to some broader theory. Yin (2009) warned that the external validity problem has been a major barrier in conducting case studies, and recommended two tactics, (as shown in Table 8), to overcome this. In the current research, three case studies were adopted, to achieve replication and theoretical (analytical) generalisation.  Reliability: if the same phenomenon is measured more than once, with the same

instrument, then the same results should be obtained (Mason, 2004). Whilst quantitative studies lead to higher levels of reliability, qualitative studies would be difficult to repeat exactly to obtain the same results (Maylor and Blackmon, 2005). Yin (2009) suggested two tactics to achieve reliability, and all are associated with the phase of data collection, (as shown in Table 8. In the current research, a case study protocol was developed, to enhance reliability. All steps taken in carrying out case studies were also clearly explained.

Reliability can be enhanced by interview training for the interviewers, and by checking interview guides or generative questions in test interviews or after the first interview (Flick, 2007). In this respect, the researcher did attend most of the training seminars or sessions held and organised by Salford Business School, and attended some research conferences, both local and international.

69

To increase the validity of the research, the interview questions were carefully prepared, piloted and refined, with the help of the researcher’s supervisor, until both the researcher and the supervisor were convinced that a satisfactory structure was in place. The literature search was conducted for the period of the past 15 years, focusing on the findings of research into the barriers to, and enablers for, sustainability in general and sustainable packaging in particular. Interview questions were designed to reflect this. Table 8 illustrates the four widely used tests and recommended case study tactics.

Table 8. Case study tactics for four design tests.

Tests Case study tactics Phase of research in

which tactics occur

Construct validity:

 Use multiple sources of evidence;  Establish chain of evidence;

 Have key informants review draft case study report.

Data collection; Data collection; Composition;

Internal validity:  Pattern matching;  Explanation-building;  Address rival explanations;  Use logic models.

Data analysis; Data analysis; Data analysis; Data analysis; External validity:  Use theory in single-case studies;

 Use replication logic in multiple-case studies.

Research design; Research design;

Reliability:  Use case study protocol;  Develop case study data base.

Data collection; Data collection. (Source: Yin (2009)).

Increasing the validity and reliability of the current research has been achieved as follows:  The researcher has generally adopted research processes that promotereliability and

validity (supported by Saunders et al., 2007);

 A multiple case study strategy, rather than a single case study strategy, has been used, to enhance reliability and validity, (supported by Yin, 2009 and Amaratunga et al., 2002);

 Multiple sources of data collection (triangulation) were used, such as semi-structured interviews, documents, direct observation and archival records, at the three organisations, in order to enhance reliability and validity (supported by Yin, 2009

70

and Saunders et al., 2009); Sutrisna (2011; 2010) argued that the validity of qualitative methods refers to the appropriateness of the method for tackling the research questions;

 The interview questions were carefully prepared and refined, with the help of the researcher’s supervisor, two experts in the field and the pilot studies that were conducted before the actual study was carried out (see Appendix 4);

 In order to promote confidence between the researcher and the interviewees, the researcher drafted a letter, authorised by his supervisor, to verify the research that was being undertaken. In addition, before starting the interview, the researcher spent time with the interviewees to explain the purpose of the study and to confirm that their personal information would remain confidential, using the information letter and consent form (see Appendix v and vi), to enhance validity and reliability, (supported by Ghauri and Gronhaug, 2005); it was also confirmed to the participants that they could withdraw from the study at any time;

 Draft transcripts of interviews were validated by respondents to confirm that they accurately represented what they had said, (supported by Ghauri and Gronhaug, 2005).