Experimental design

Experiments are causal designs and the key strengths of this technique are the

identification of causal connections and the capacity to distinguish between causes and the observed and measurable effects (Churchill and Iacobucci, 2005). This technique lends itself to the rigorous testing of hypotheses and in addition, enables the use of pictures or advertisements, which help make tasks more meaningful, or closer to “reality”. This is

hugely advantageous to the present research as the hypothesis testing necessitates a controlled exposure of participants to advertisements that manipulate specific variables.

They key advantage of an experiment is that it has a greater ability to supply evidence of causality because of the control it affords researchers. Because researchers are able to control at least some manipulations of the presumed causal factor, they can be more confident that the relationships discovered are “true” relationships (Churchill and Iacobucci, 2005). Given that an experiment is capable of providing evidence of causal relationships and enables the use of advertising stimuli in its procedure, it is deemed the most

appropriate method for the research at hand.

Shadish et al (2002) identify four different types of causal methods which range from the

“gold standard” (Shadish et al, 2002, p13) of a randomized experiment, to

quasi-experiments, natural experiments and correlations. The key differences between these are presented in table 4 below. An experiment is a type of study where a treatment is

deliberately introduced in order to observe it’s effects. By definition they are orthogonal as the treatments are varied independently from one another which makes it easier to isolate the effect of the treatment on the observed responses, therefore avoiding multicollinearity between treatments. In order to conduct an experiment the hypotheses must be testable and three conditions must be met. First, that there are procedures for manipulating the setting. Second, the predicted outcome must be observable, and finally the predicted outcome must be measurable (Myers and Hansen, 2012).

Type Definition

Randomised experiment

An experiment in which units are assigned to receive the treatment or the alternative condition by a random process such as the toss of a coin or a table of random numbers.

Quasi- experiment An experiment in which units are not assigned to conditions randomly

Natural experiment Adaption of an experiment as the cause cannot be manipulated e.g. a study that contrasts a naturally

occurring event such as an earthquake with a comparison condition

Correlational study Usually synonymous with non-experimental or

observational study, a study that simply observes the size and direction of a relationship between variables

Table 4 - Adapted from Shadish et al (2002, p12)

Randomised experiments are where the experimental unit (e.g. people, time period or institution) are assigned to a treatment by chance (Shadish et al, 2002). A minimum of two groups are created which are probabilistically similar to each other on average, which means that the outcomes of observed differences between the groups are likely to be an effect of the treatment, as opposed to individual differences between units. Quasi- experiments are similar to randomized experiments but are lacking one of the essential elements (e.g. manipulation of antecedents or random assignment of units to treatments) (Myers and Hansen, 2012). Indeed, the differences between randomized experiments and quasi-experiments can be subtle (Field and Hole, 2003). Often quasi- experimental groups are based on the event, characteristic or behavior whose influence is under investigation and units either self-select the treatment they are assigned to or an administrator to the process assigns units to treatments (Shadish et al, 2002). This is a key distinction because this means that the differences between groups may be systematic (i.e. non-random) and therefore researchers must rule out plausible alternative explanations of any observed and measured effect.

Natural experiments are defined by the research context within which the causal relationship under examination occurs. These methods observe a “naturally-occurring contrast between a treatment and a comparison condition” (Shadish et al, 2002, p17). The treatments themselves are not malleable and often are events or phenomena such as earthquakes or terrorist attacks. When experiments are not possible due to practical or ethical reasons, correlation studies can be employed to compare relationships or associations between variables (Myers and Hansen, 2012). Such methods differ from experiments where the objective is to identify differences between treatments, as the associations between variables are examined to identify cause and effect relationships.

Thus, whilst cause and effects are measures the structural elements of experiments (e.g.

pre-tests, random assignment to treatments) are absent. Generally speaking, this leads to problematic issues for researchers, for example in a cross sectional study because the data is collected at one point in time it is difficult to assure that cause precedes effect (Shadish et al, 2002).

Given the research hypotheses outlined in Chapter 4 and the objectives of the present study, natural experiments and correlation studies are not suitable methods to observe the cause and effect relationship between threat advertisement manipulations/ treatments and emotional, cognitive and behavioural effects of those treatments. The most appropriate method is a randomised experiment where participants are allocated to treatments by chance. This allows for a confident attribution of differences in observed effects to the

treatment variables, which is not possible with quasi-experiments. Indeed, the use of randomised experiments allows for “control by design” (Keppel and Wickens, 2004, p7) whereby the design of the experiment naturally controls for individual differences and nuisance factors.

5.2.1 Factorial experimental design

As outlined in the conceptualisation and hypothesised relationships between variables in Chapter 4, this research examines the effects of three independent variables manipulated in threat advertisement treatments on emotion, cognition and behavior variables. A

randomised experiment is the most appropriate method to collect data to measure the differences between treatments. Research designs where two or more independent variables are studied at the same time are factorial designs (Myers and Hansen, 2012) where every level of one factor is combined with every level of the other(s). Indeed, factorial designs are rich with information because they involve the variation of multiple independent variables within a single study (Keppel and Wickens, 2004).

There are three general factorial designs (Hair et al, 2006). First is the between-subjects factorial design, in which every treatment is assigned to a different sample of units.

Alternatively there is the within-subject factorial design, in which a single sample of units is assigned to every experimental treatment. Mixed designs adopt some within-subjects factors and some between-subjects factors. This type of design combines the advantages (and, it must be said, disadvantages) of between-subjects and within-subjects designs (Keppel and Wickens, 2004). There are distinct advantages and disadvantages associated with the three different types of factorial design which are outlined in table 5 below.

Whilst between-subjects designs have practical benefits, such as ease of design and analysis, they are less sensitive and therefore require a large number of units. However, whilst within-subjects designs allow for direct comparison between treatments among the same sample, nuisance variables such as order effects must be accounted for. Equally when a large number of variables are to be examined, a within-subjects design becomes cumbersome and inefficient. (Keppel and Wickens, 2004). A mixed design can be utlilised to simplify overly complex designs which necessitate a large number of between-subjects treatments or reduce the likelihood of respondent fatigue prompted by a lengthy within-subjects design.

Between subjects

Disadvantages 1. Samples are less sensitive.

Table 5 - Comparison of factorial designs

As there are three independent variables for this study, a between subjects factorial design is most appropriate because of the efficiency of the design and the requirement of the smallest number of statistical assumptions. Thus, a 2x2x2 between subjects factorial design is employed, with the graphic nature of the image (graphic and no graphic), the message frame (loss or loss avoidance) and the direction of threat (self or other) manipulated between-subjects. This leads to eight experimental treatment conditions. Whilst there are advantages to a within-subject design, this approach would have been too demanding for the respondents who would have been required to evaluate a large number of

advertisement stimuli containing the experimental manipulations, almost certainly triggering respondent fatigue. A between subjects design enables the researcher to keep the

complexity of the experimental design at a manageable level with eight sub groups of

respondents and to minimise respondent fatigue, while being able to test all the hypotheses of the conceptual framework. Furthermore, a large number of subjects were recruited (as outlined in section 5.7) in order to counterbalance the lack of sensitivity that characterises between-subject designs (Keppel and Wickens, 2004). The eight between-subjects treatments are outlined in table 6 below.

Experimental Condition

Between-subjects factors

Message Frame Message Direction Graphic Image

1 Loss Avoidance Other Graphic

2 Loss Avoidance Other No Graphic

3 Loss Avoidance Self Graphic

4 Loss Avoidance Self No Graphic

5 Loss Other Graphic

6 Loss Other No Graphic

7 Loss Self Graphic

8 Loss Self No Graphic

Table 6 - Factorial design of experimental treatments