The Underlying Design Logic

Figure 7 – Designing the Logic Underlying the Research

A. Pre-Structured Design

The study follows a pre-structured or “tightly coordinated” (Miles and Huberman 1994; Vella and Foxall 2011; Miles et al. 2013; Vella and Foxall 2013) rule-driven (Yin 2014) design with an explicitly stated methodology that maps out and guides the entire research process and that includes the various procedures to ensure the validity and reliability of the research including the fair and accurate collection and treatment of evidence (Vella and Foxall 2011, 2013). The design mitigates against such threats as lack of clearly delineated research bounds, the loss of focus and clarity, being overloaded by the data, the danger of not completing research projects on time (Dnes 1992; Miles and Huberman 1994; Patton 2002; Miles et al. 2013; Yin 2014), and situations where the evidence collected does not answer the original research questions (Yin 2014)⁶⁹.

The pre-structured design adopted allows for minor changes in the design to suit the unanticipated events emerging from data collection without altering the scope of the research (Yin 2014). In addition, during the early stages of the research process, iterating between theory and data aided significantly in refining certain aspects of design specifically in establishing the

69 Qualitative research is usually associated with research processes that are relatively loose and unstructured with the advantage of a significant degree of flexibility and sensitivity to unanticipated issues that emerge when the research is in progress (e.g., Miles and Huberman 1994; Lee 1999; Mason 2002; Gephart 2004; Maxwell 2005; Bryman and Bell 2007; Miles et al.

2013). For a detailed exposition of such designs refer to Chapter 1 of Maxwell (2005) and the book, Qualitative Researching, by Mason (2002). Maxwell strongly advocates a loose design on the basis of linear and sequential designs running counter to the principle of ongoing reflexivity that underpins qualitative (interpretivistic) research. However, the components around which Maxwell builds his approach are very similar to titghtly coordinated and structured designs.

unit of analysis and embedded sub-units and in fine-tuning the case focus and the boundaries of the study⁷⁰.

More importantly, pre-structured designs are prone to the possibility of

“bending data out of contextual shape” when answering analytical questions (Miles et al. 2013, p. 20). A number of precautions were adopted to ensure the validity and reliability of the research and eliminate the risk of bending the data to fit the theoretical elements that interlace the pre-structured design when performing case analysis. These include: iterating between theory and data (Section 2.3.2), screening several alternative secondary reports according to certain criteria (Section 2.3.3), and piloting an initial case focus and boundaries on the evidence prior to conducting a final analysis (Vella and Foxall 2011).

B. Longitudinal Design

The study of such causal processes, dynamics, and their outcomes dictates a longitudinal design (Aldrich and Pfeffer 1976; Pettigrew 1990; Lee 1999; Morlacchi and Nelson 2011; Bairstow and Young 2012; Lavie and Singh 2012; Yin 2014) irrespective of whether research aims to generate or test theory⁷¹. A longitudinal design focuses on analysing and tracking historical, non-linear processes of change embedded within particular contexts (Pettigrew 1990; Langley et al. 2007). Evolutionary research shares an emphasis on dynamics (i.e., non-linear processes) and, thus, on the importance of historical reconstruction (Coriat and Dosi 1998; Dosi and Marengo 2007; Dosi 2013).

Given these considerations and the aims of the project, the research is organised as a longitudinal single case study (Yin 2014).

C. Single versus Multiple Cases

The consensus among methodologists favours the use of multiple cases, where possible, to generate more robust conclusions through structured

comparisons and deeper theoretical contributions through the extent of

70 According to Maxwell (2005), a linear and sequential model that eliminates flexibility with respect to unanticipated issues. Similarly Miles et al. (2013) claim that pre-structured designs significantly reduce sensitivity to case specifics.

71 A longitudinal design is defined as any design that requires collecting and analyzing data from a population at different time intervals to facilitate temporal comparisons (Babbie 1990; Bryman and Bell 2007; Gerring 2012).

replication across different cases (e.g., Eisenhardt 1989; George and Bennett 2005; Miles et al. 2013; Yin 2014; cf. Dyer Jr and Wilkins 1991). Ideally, causal processes are examined and established across multiple cases to generate a deeper and more powerful description and interpretation (Miles et al. 2013).

Adopting a multiple case study approach to accomplishing the present research objectives would have generated a significant degree of complexity due to the sheer volume of historical qualitative data that would have needed consideration, analysis, interpretation, and reporting. Timely completion of the projected would have been threatened. Despite the important analytical

advantages of multiple case studies, the single case study design was chosen⁷².

A signal limitation of single case studies is that the processes observed therein may be unique to the case and, therefore, any support to theoretical claims may be idiosyncratic (Vella and Foxall 2011; Miles et al. 2013; Vella and Foxall 2013; Yin 2014). Cross-case comparisons are not possible and therefore generalizability outside the cases studied is very limited. Indeed, only analytic generalisation may be invoked (Lee 1999; Vella and Foxall 2011; Miles et al.

2013; Vella and Foxall 2013). Obviously, theoretical replication of the results across multiple cases is not possible and this may weaken the status of

analytical conclusions (Miles et al. 2013; Yin 2014). Single case studies run a greater risk of being judged indeterminate when faced by possible rival

explanations and of leading to incorrect conclusions due to errors in

measurement (George and Bennett 2005). This said, however, single case studies may be considered as pilots assessing the feasibility of conducting future multiple case study research with identical research objectives for the purpose of replication, comparison, (George and Bennett 2005; Yin 2014) and falsification.

Despite these limitations several researchers have successfully utilised single case studies (Dyer Jr and Wilkins 1991; George and Bennett 2005; Yin 2014). In addition, causal relations may still be demonstrated through the use

72 Appendix A2.3 describes the three options of multiple case study designs considered during the initial stages of the research.

of single rather than multiple cases via analytical techniques that rely on theory and research propositions developed a priori (George and Bennett 2005).

D. Steady State Baseline and Replication Logic

To capture some of the advantages of multiple case study design, the case is set up as if it were a steady state experiment to allow within-study literal replication. The objective of this design is to allow the procedure of operant conditioning to be demonstrated qualitatively with a reduced reliance on inference.

The design is based on one of the two main EAB approaches to studying how experimental conditions effect response rates. Initially, several

measurements of behaviour are collected under the baseline condition with the expectation being that ultimately a relatively stable pattern of responding

emerges. This baseline reflects the typical influence of the baseline condition on the rate of responding. The baseline is the “steady or stable state of responding … defined as … a pattern of responding that shows little relative variation in its measured dimensional quantities over some period of time”

(Johnston and Pennypacker 2009, p. 196). The main properties of the steady state is that it is a relative measure of stability (or variation): the stability is attributable only to dimension of behaviour that is being measured (in this case the rate of response), and, any extraneous environmental variables have been either accounted for or found to be weak. Similar measures are collected under the treatment condition. The procedure involves measuring and comparing the behaviour of a single subject under control or baseline and treatment conditions repeatedly (Johnston and Pennypacker 2009). This steady state approach, therefore, (a) allows the accumulation of data on the interaction of behaviour with the baseline condition; and, (b) it provides the foundation to make comparisons between the baseline and treatment conditions (Johnston and Pennypacker 2009)⁷³. In other words, it allows measuring changes in the rates of responding as the environment changes.

73 Reference is made to Johnston and Pennypacker (2009) for more details on the approach.

The Commission report is set up in a manner that emulates this steady state approach⁷⁴. Following this procedure, the research sets the two

generation-situations (Figure 5, the situation is the unit of analysis⁷⁵) as if they were distinct and successive experiments and examines the degree of similarity and dissimilarity across cases. Literal replication of results from one case to the next should result in similarities according to the predictions of the operant theory and, therefore, analytic or theoretical generalisation rather than statistical generalisation may be accomplished (Yin 2014).

The proposed design postulates the first generation-situation faced by Wall’s during the period 1922 to 1969 as the baseline and the second

generation-situation (1970 to ca.1978) as being analogous to the treatment condition. The analysis traces how and the extent to which this baseline remains stable and varies over the treatment condition (noting variation,

novelty, and retained and discontinued practices). The baselines extracted are (a) a reconstruction of the learning history of Wall’s until 1969 and (b) the character of extant environmental conditions until the same period. Both are treated as relatively stable states that emerged as a result of earlier processes of selection⁷⁶. Stability (hence, replicability) and change in both baselines are traced through the analysis of the subsequent generation. The extent of replicability will demonstrate the extent to which practices continue, recur, persist, or are discontinued over the years. Once practices emerge repeatedly in the course of the history of Wall’s through its environmental interaction, it may

74 The reason that the design emulates rather than replicates the logic is due to differences that exist between experimental laboratory settings and real world complex contexts of behaviour.

The EAB exercises significant control over independent variables and over any other elements within the laboratory setting that may confound results and thus invalidate the experiment (e.g.

Catania 1999; Cooper et al. 2007; Pierce and Cheney 2008; Johnston and Pennypacker 2009).

In addition, this control and quantitative measurement provide EAB researchers with a clear route to identify and objectively observe antecedent stimuli, behaviour and consequences and the effects of any inter-relationships among them (Foxall 2010b). Within non-experimental designs it is significantly more difficult to: (a) identify and account for all the elements of the three-term contingency including and especially the learning history of the individual; and, (b) unequivocally establish inter-relationships among the elements (Foxall 1990, 1994, 1998b, 2010b). The number, salience, and the complexities of the contingencies in operation in a given context also impose limits on interpretations (Lee 1988) to the extent that some important contingencies may be invisible to researchers. Lee (1988) highlights an additional complication:

interpretations are a function of the evidence available at hand. A lack of information may hamper the identification of the consequences of behaviour and in constructing the

contingencies governing the behaviour of research subjects. (See also Appendix 2, Section A2.6.3B).

75 Section 2.3.4 discusses the unit of analysis.

76 The notion that the baseline represents a steady state should not be mistaken as implying an equilibrium state.

be possible to infer a process analogous to operant conditioning and to selective retention and elimination.