Evaluation Criteria

Two sets of evaluation criteria are adopted: (1) plausibility, usefulness, and objectivity as predicated by radical behaviourist methodology, and, (2) validity and reliability as these apply to positivist case study research and within the narrow scope of the meaning the term “qualitative” takes within this

research (Vella and Foxall 2011)⁹⁹.

A. Plausibility, Usefulness and Objectivity

Since plausibility lacks proper definition (Foxall 2010b), the criterion is understood within this research to refer to: (a) the extent to which the

conclusions and interpretations drawn are credible and persuasive; and, (b) more importantly, the degree to which these interpretations (and the

frameworks upon which these are built) are logically consistent with both operant principles and the domain within which these principles are applied (Vella and Foxall 2011).

With respect to the plausibility of theoretical frameworks, two additional considerations are made: (a) the extent to which the frameworks may be operationalized and may generate propositions or hypotheses for application, evaluation, and testing in empirical research; and, (b) the capacity of the frameworks to link theory and empirical observation comprehensively and intelligibly (i.e., a capacity to exhaustively capture the phenomenon frameworks have been designed to interpret and explain) (e.g. Foxall 2010b). Plausibility is

98 The manner in which Vella and Foxall (2011) analyse the evidence suggests additional strategies for conducting interpretations. Other techniques adopted include data reduction (coding, data displays, main elements of the SMC framework), within-case analysis and the drawing and verifying conclusions (Miles et al. 2013). The analytical and interpretative process is essentially iterative (Vella and Foxall 2011; Miles et al. 2013). Appendix A2.6.2 describes and elaborates on the additional strategies used by Vella and Foxall (2011).

Appendix A2.6.3 details the techniques involved in data reduction, within-case analysis, and conclusion drawing and verification following, on the most part, the prescriptions of Miles et al.

(2013), a standard text in the analysis of qualitative data.

99 See Appendix A1.2 for an explanation of what is meant by qualitative within this research.

There are several criteria predicated by the various epistemological stances for judging the quality of research (Guba and Lincoln 1994; Mason 2002; Patton 2002; Bryman and Bell 2007;

Gibbert et al. 2008; Miles et al. 2013). The discussion within this section focuses on criteria relevant to the assumed radical behaviourist position and with emphasis on falsification.

presumably enhanced when researchers express greater confidence with respect to these two dimensions. In addition, the use of the BPM improves plausibility because the model explicitly takes into account differences in behaviour within experimental laboratories in contrast to the real world and the idiosyncrasies of human learning (e.g. Foxall 2010b). Plausibility is enhanced by treating the components of the BPM as sensitizing concepts and the

empirical and theoretical conclusions derived from research as working hypotheses. Components and conclusions are continually subjected to

structured and focused empirical testing, to gradual refinements via additional real world observation and theoretical considerations, and, to peer review and evaluation.

Given the trade-off between plausibility, on the one hand, and external validity and reliability, on the other, the research emphasises the latter criteria (Staddon 2001).

Usefulness is understood in terms of the adequacy of the sensitizing framework in rendering a plausible, valid, and reliable operant interpretation of firm behaviour.

Since interpretation is theory-laden and subjective to varying degrees, the criterion of objectivity requires an attempt to be as impartial, neutral, and unbiased as possible (Vella and Foxall 2011)¹⁰⁰.

B: Validity and Reliability

The four criteria of quality governing positivist case study research are construct validity, internal validity, external validity, and reliability (Gibbert et al.

2008; Yin 2014)¹⁰¹.

100 As a former marketing practitioner, special attention is awarded to the issue of value-ladenness and related biases (Bryman and Bell 2007) that may arise due to knowledge and experience in the conduct of business organisations. Certain practices that are considered as standard conduct in business may be regarded as unethical by social scientists (Vella 2010).

Although the experience is invaluable in rendering the interpretation and discussion more plausible, exercising a degree of reflexivity is important. The research does not take a normative stance with respect to the ethical issues raised by any monopolistic behaviour identified and studied.

101 Although Vella and Foxall (2011, 2013) are concerned with validity and reliability, they do not specify what specific criteria they utilize to evaluate their study.

Several tactics are available to combat issues with respect to construct validity¹⁰²: To avoid confirmation bias, Flyvbjerg (2011) suggests explicitly recognising sources of researcher bias, assumptions and preconceptions and, following the analysis, discussing these together with concepts and hypotheses in the light of empirical evidence. The discussion should point to empirical evidence that appears to falsify assumptions, concepts, and hypotheses (Flyvbjerg 2011). The use of rival explanations, where possible, contributes towards reducing confirmation bias. In addition, the search for empirically grounded alternative rival explanations highlights the characteristic tendency of case studies towards falsification rather than verification (Flyvbjerg 2011; Yin 2014).

Using multiple sources of evidence and other forms of triangulation are techniques for combatting verification bias (Gibbert et al. 2008; Yin 2014) and for refining concepts (Eisenhardt 1989; Gibbert et al. 2008; Yin 2014). Using evidence produced by a third party avoids the verification biases of the author of this research. The adequate level of triangulation in the Commission’s report mitigates the biases of the original investigators. Yin (2014) suggests stating the phenomenon of interest around a set of very specific concepts and using operational definitions and measures that already exist in the literature if applicable. This ensures adequate operationalization (Yin 2014). The overarching theory led case study design, the use of the BPM, the iterative strategy, and the specification of constructs and measures are therefore presumed to enhance construct validity. In addition Mason (2002) suggests that demonstrating validity also requires researchers to show that the chosen method, collection instruments and sources of data are adequate to capture and reflect the various theoretical concepts of interest. Other research quality improvement strategies include: (1) designing and implementing a robust and well-conceived theory-led data analysis strategy aimed at enhancing

measurement validity and reliability (Potter and Levine-Donnerstein 1999); and, (2) explicitly stating the logic underlying all methodological choices and

102 Construct validity is here defined in relation to concerns with respect to whether concepts are sufficiently defined to allow operationalization and proper investigation (Eisenhardt 1989;

Gibbert et al. 2008; Yin 2014) and whether the researcher introduces verification or confirmation bias during instrumentation and consequent data collection (Flyvbjerg 2011; Yin 2014).

demonstrating how the analysis and interpretation have been constructed (Mason 2002).

Attending to empirically based rival explanations as systematic search for causality is an analytic strategy aimed at jointly minimising the threats of internal validity (George and Bennett 2005; Yin 2014)¹⁰³. An explicit theoretical

framework for conducting the research also enhances internal validity (Gibbert et al. 2008). Comparisons between empirical evidence and theory within the discussion improve the sharpness of constructs thereby improving internal validity (Eisenhardt 1989).

With respect to external validity, the research aimed only at achieving analytic generalisation, i.e., whether theoretical propositions and concepts fit, partially fit or do not fit with the empirical observations contained in the case (Lee 1999, p. 157)¹⁰⁴.

103 Internal Validity deals with (a) concerns of drawing inferences when events are either not observed directly (Yin 2014) or not completely represented (e.g., through omission, different research objectives driving the compilation and collection of the secondary dataset and so on);

and, (b) the extent to which claims of causal relations between independent and dependent variables may be warranted (Bryman and Bell 2007; Gibbert et al. 2008; Yin 2014). In qualitative (interpretivist) research, internal validity is paralleled by the credibility criterion, i.e., the extent to which interpretation is trustworthy and believable (Guba and Lincoln 1994; Lee 1999; Mason 2002; Bryman and Bell 2007; Miles et al. 2013). By definition, therefore, there is a degree of overlap with the plausibility criterion.

104 External Validity deals with the issue of whether the conclusions of the research are generalizable beyond the confines of the immediate work (Bryman and Bell 2007; Yin 2014).

Flyvbjerg (2011), for example, addresses the misrepresentation of generalizability as a threat to the integrity of the case study as a rigorous research method. Whereas the use of multiple cases does enhance the potential for generalizability of the findings (e.g., Eisenhardt 1989;

George and Bennett 2005; Gerring 2007; Yin 2014), generalization from a single case is severely limited irrespective of its design and sampling methods (cf. Flyvbjerg 2011). This limitation, however, does not threaten the case study either as a rigorous research method in its own right or as a means of generating valuable insights and constructing theory.

In application to case research, Yin (2014) uses reliability to refer to the extent of repeatability or replicability of the research¹⁰⁵. In this sense, creating an audit trail tracing (and justifying) all the methodological steps and choices increases the potential of the research being repeated and later investigators arriving to the same conclusions (Bryman and Bell 2007; Gibbert et al. 2008;

Vella and Foxall 2011, 2013; Yin 2014). The audit trail is created through the case study protocol¹⁰⁶, the case database (Yin 2014), and documenting the entire case design (Vella and Foxall 2011, 2013). This encourages complete transparency by facilitating comprehensive retrieval of the data and

reproducibility of the study (Gibbert et al. 2008). Replicability also aims minimising bias and error within a study (Gibbert et al. 2008; Yin 2014) and steps have been taken to reduce these within reasonable limits.

Having established the design of the case study, the following chapter presents two theoretical building blocks – (1) a general understanding of the central principles of operant psychology, and, (2) a critical exploration of Selection by Consequences. The BPM, the Marketing Firm, and the SMC are explained in Chapter 4.

105 In general, however, there are limitations with respect to replicability of qualitative research:

for example, the impossibility of faithfully reproducing particular social situations (Bryman and Bell 2007, p. 410) or duplicating all the conclusions drawn. While the use of secondary data in the public domain gives rise to the opportunity for reproducing the study, this is not entirely possible because of a degree of subjective judgement introduced by different researchers (and their disparate experiences and skills). Mason (2002) interprets reliability in application to qualitative research to retain some of the meaning the term holds in quantitative research: That is, to refer to issues with respect to the consistency and accuracy of method and technique in application and reapplication. It is not possible to attend to reliability in the same manner as with quantitative research. However, Mason (2002) suggests that researchers must

demonstrate that the data collection, generation and analytic methods used are the more appropriate means to addressing the research objectives and answering questions as accurately as possible. For example, evidence should not be neglected, misrepresented, invented, or treated unfairly. In addition, conclusions drawn must be warranted through

carefully constructed explanations (Mason 2002; Miles et al. 2013). Therefore, the creation and maintenance of an audit trail, attention to possible rival explanations, the development of a case study protocol (Appendix A2.5), the adoption of standardised data collection instruments, and the application of coding rules/techniques are all aimed to enhance the accuracy and

consistency of the study. In addition, reliability is improved by relying on the general

methodology provided by Vella and Foxall (2011, 2013) and on the specified tactics for drawing and verifying conclusions.

106 See Appendix A2.5.

85