We have now identified a basic problem. Researchers simply do not and cannot know just what other variables may affect their key measures. Is there any way in which all confounding variables can be taken into account when we do not know what those vari-ables are? For some psychologists, the answer to the major problems of research design lies in the process of randomisation. Basically we would form two groups of participants who are given the opportunity to interact in pairs and get to know each other better. In one condition, the pairs are formed by choosing one member of the pair at random and then the other member selected at random from the participants who had similar attitudes to the first member of the pair. In the other condition, participants are selected at random but paired with another person dissimilar in attitude to them, again selected at random.
By allocating participants to similarity and dissimilarity conditions by chance, any dif-ferences between the conditions cannot be accounted for by these confounding variables.
By randomising in this way, similarities and dissimilarities in the areas from which the participants come, for example, would be expected to be equalised between groups.
This particular example is illustrated in Figure 1.11 and the more general principles of experimental design in Figure 1.12.
The simplest way of randomisation in this example is to allocate participants to the different conditions by tossing a coin. We would have to specify before we tossed the coin whether a coin landing heads facing upwards would mean that the person was paired with a person with the same attitude as them or with a different attitude from them. If we tossed a coin a fixed number of times, say 20 times, then it should come up heads 10 times
FIGURE 1.11 The experimental design to investigate attitude similarity and friendship
and tails 10 times on average. If we had decided that a head means meeting someone with the same attitude, approximately 10 people will have been chosen to meet some-one with the same attitude as them and approximately 10 somesome-one with a different attitude from them. This kind of procedure is known as random assignment. People are randomly assigned to different situations which are usually called conditions, groups or treatments. (Actually we have not solved all of the difficulties as we will see later.)
If half the people in our study came from, say, Bristol and half from Birmingham, then about half of the people who were randomly assigned to meeting a person with the same attitude as them would be from Bristol and the remaining half would be from Birmingham, approximately. The same would be true of the people who were randomly assigned to meeting a person with a different attitude from them. About half would be from Bristol and the remaining half would be from Birmingham, approximately. In other words, random assignment should control for the area that people come from by ensuring that there are roughly equal numbers of people from those areas in the two groups. This will hold true for any factor such as the age of the person or their gender.
In other words, random assignment ensures that all confounding factors are held con-stant – without the researcher needing to know what those confounding factors are.
Sampling error
The randomised study is not foolproof. Sampling error will always be a problem. If a coin is tossed any number of times, it will not always come up heads half the time and tails half the time. It could vary from one extreme of no heads to the other extreme of
FIGURE 1.12 The general principles of experimental design
all heads, with the most common number of heads being half or close to half. In other words, the proportion of heads will differ from the number expected by chance. This variability is known as sampling error and is a feature of any study. A sample is the num-ber of people (or units) that are being studied. The smaller that the sample is, the greater the sampling error will be. A sample of 10 people will have a greater sampling error than one of 20 people. Although you may find doing this a little tedious, you could check this for yourself in the following way. Toss a coin 10 times and count the number of heads (this is the first sample). Repeat this process, say 30 times in total, which gives you 30 separate samples of coin tossing. Note the number of times heads comes up for each sample. Now do this again but toss the coin 20 times on each occasion rather than 10 times for each sample. You will find that the number of heads is usually closer to half when tossing the coin 20 times on each occasion rather than 10 times (see Figure 1.13).
Many studies will have as few as 20 people in each group or condition because it is thought that the sampling error for such numbers is acceptable. See our companion statistics text, Introduction to Statistics in Psychology (Howitt and Cramer, 2011a) for a more detailed discussion of sampling error.
The intervention or manipulation
So, in many ways, if the purpose of one’s research is to establish whether two variables are causally related, it is attractive to consider controlling for confounding variables through random assignment of participants to different conditions. To determine whether similar attitudes lead to friendship, we could randomly assign people to meet strangers with either similar or dissimilar attitudes to themselves as we have already described.
Remember that we have also raised the possibility that people’s attitudes are related to
FIGURE 1.13 A sampling ‘experiment’
other factors such as the area or kind of area they come from. Assuming that this is the case, participants meeting strangers with the same attitudes as themselves might be meeting people who come from the same area or kind of area as themselves. On the other hand, participants meeting strangers with different attitudes from them may well be meeting people who come from a different area or kind of area to themselves. In other words, we still cannot separate out the effects of having the attitude similarity from the possible confounding effects of area similarity. It is clear that we need to disentangle these two different but interrelated factors. It is not possible to do this using real strangers because we cannot separate the stranger from the place they come from.
Let’s consider possible approaches to this difficulty. We need to ensure that the stranger expresses similar attitudes to the participant in the same attitudes condition.
That is, if they did not share attitudes with a particular participant, they would never-theless pretend that they did. In the different attitudes condition, then, we could ensure that the stranger always expresses different attitudes from those of the participant. That is, the stranger pretends to have different attitudes from the participant. See Table 1.1 for an overview of this. In effect, the stranger is now the accomplice, confederate, stooge or co-worker of the researcher with this research design.
The number of times the stranger does not have to act as if they have a different attitude from the one they have is likely to be the same or similar in the two conditions – that is, if participants have been randomly allocated to them. This will also be true for the number of times the stranger has to act as if their attitude is different from the one they have.
Unfortunately, all that has been achieved by this is an increase in complexity of the research design for no other certain gain. We simply have not solved the basic problem of separating similarity of attitude from area. This is because in the same attitude condition some of the strangers who share the same attitude as the participant may well be attractive to the participant actually because they come from the same area as the participant – for example, they may speak with similar accents. Similarly, some of the participants in the different attitudes condition will not be so attracted to the stranger because the stranger comes from a different area. Quite how this will affect the outcome of the research cannot be known. However, the fact that we do not know means that we cannot assess the causal influence of attitude similarity on attraction with absolute certainty.
We need to try to remove any potential influence of place entirely or include it as a variable in the study. Probably the only way to remove the influence of place entirely is by not using a real person as the stranger. One could present information about a stranger’s attitude and ask the participant how likely they are to like someone like that.
This kind of situation might appear rather contrived or artificial. We could try to make it less so by using some sort of cover story such as saying that we are interested in finding out how people make judgements or form impressions about other people. Obviously the participants would not be told the proposition that we are testing in case their behaviour is affected by being told. For example, they may simply act in accordance with their beliefs about whether or not people are attracted to others with similar attitudes.
Not telling them, however, does not mean that the participants do not come to their own Table 1.1 Manipulating similarity of attitude
Condition Participant Stranger
Same attitude Same as stranger No acting
Different from stranger Act as if the same Different attitude Same as stranger Act as if different
Different from stranger No acting
conclusions about what the idea behind the study is likely to be and, perhaps, act accordingly.
What we are interested in testing may not be so apparent to the participants because they take part in only one of the two conditions of the study. Consequently they are not so likely to realise what was happening (unless they talked to other people who had already participated in the other condition of the study). We could further disguise the purpose of our study by providing a lot more information about the stranger over and above their attitudes. This additional information would be the same for the stranger in both conditions – the only difference is in terms of the information concerning attitude similarity. In one condition attitudes would be the same as those of the participant while in the other condition they would be different.
If (a) the only difference between the two conditions is whether the stranger’s attitudes are similar or dissimilar to those of the participant and (b) we find that participants are more attracted to strangers with similar than with dissimilar attitudes then this differ-ence in attraction must be due to the only differdiffer-ence between the two conditions, that is, the influence of the difference in attitudes. Even then there are problems in terms of how to interpret the evidence. One possibility is that the difference in attraction is not directly due to differences in attitudes themselves but to factors which participants associate with differences in attitudes. For example, participants may believe that people with the same attitudes as themselves may be more likely to come from the same kind of area or be of the same age. Thus it would be these beliefs which are responsible for the differences in attraction. In other words when we manipulate a variable in a study we may, in fact, inadvertently manipulate other variables without realising it. We could try to hold these other factors constant by making sure that the stranger was similar to the participant in these respects, or we could test for the effects of these other factors by manipulating them as well as similarity of attitude.
This kind of study where:
z the presumed cause of an effect is manipulated, z participants are randomly assigned to conditions, and z all other factors are held constant
was called a true experiment by Campbell and Stanley (1963). In the latest revision of their book, the term ‘true’ has been replaced by ‘randomised’ (Shadish, Cook and Campbell, 2002, p. 12). If any of the above three requirements do not hold then the study may be described as a non-experiment or quasi-experiment. These terms will be used in this book. True or randomised experiments are more common in the sub-disciplines of perception, learning, memory and biological psychology where it is easier to manipulate the variables of interest. The main attraction of true experiments is that they can provide logically more convincing evidence of the causal impact of one variable on another. There are disadvantages which may be very apparent in some fields of psychology. For example, the manipulation of variables may result in very contrived and implausible situations as was the case in our example. Furthermore, exactly what the nature of the manipulation of variables has achieved may not always be clear. Studies are often conducted to try to rule out or to put forward plausible alternative inter-pretations or explanations of a particular finding. These are generally beneficial to the development of knowledge in that field of research. We will have more confidence in a research finding if it has been confirmed or replicated a number of times, by different people, using different methods and adopting a critical approach.
It should be clear by now that the legitimacy of assertions about causal effects depends on the research design that has been used to study them. If we read claims that a causal effect has been established, then we might be more convinced if we find that the studies in question which showed this effect were true experiments rather than quasi-experiments.
Furthermore, how effectively the causal variable was manipulated also needs to be con-sidered. Is it possible, as we have seen, that other variables were inadvertently varied at the same time? The nature of the design and of any manipulations that have been carried out are described in journal articles in the section entitled ‘Method’.
These and other designs are discussed in more detail in subsequent chapters. Few areas of research have a single dominant method. However, certain methods are more characteristic of certain fields of psychology than others. The results of a survey of a random sample of 200 studies published in the electronic bibliographic database PsycINFO in 1999 (Bodner, 2006) revealed that a variety of research designs are common but dominated by experimental studies. The findings are summarised in Figure 1.14.
Studies investigating the content of psychology journals are not frequent and this is the most recent one. Knowing about the strengths and weaknesses of research designs should help you to be in a better position to critically evaluate their findings. There is more on design considerations in later chapters. A comparison of the main research designs is given in Figure 1.15.
FIGURE 1.14 Different types of design in 200 PsycINFO articles
FIGURE 1.15 The major advantages and disadvantages of the main research designs
1.6 Practice
Psychologists believe in the importance of the empirical testing of research ideas. Con-sequently, doing research is a requirement of most degrees in psychology. For example, to be recognised by the British Psychological Society as a practising psychologist you need to show that you have a basic understanding of research methodology and the skills to carry it out. This is the case even if you do not intend to carry out research in your profession. Training in research is an important part of the training of most practitioners such as educational and clinical psychologists. Practising psychologists simply cannot rely on academic psychologists to research all of the topics from which psychological practice might benefit. The concept of practitioner–researcher has developed in recent years. This is the idea that practitioners such as occupational psychologists and forensic psychologists have a responsibility to carry out research to advance practice in their field of work. To be brutally frank, a student who is not prepared to develop their research skills is doing themselves and the discipline of psychology no favours at all.
1.7 Conclusion
Most psychological ideas develop in relation to empirical data. Propositions are made, tested and emerge through the process of collecting and analysing data. The crucial activity of psychologists is the dissemination of ideas and findings which emerge largely through empirical work in the many fields of psychology. The prime location to find such developments and ideas is in the academic and practitioner journals which describe the outcomes of psychological research. Other important contexts for this are academic and practitioner conferences geared to the presentation of ongoing research develop-ments in psychology and, to a lesser degree, academic books. These various forms of publication and presentation serve a dual purpose:
z To keep psychologists abreast with the latest thinking and developments in their fields of activity.
z To provide psychologists with detailed accounts of developing research ideas and theory so that they may question and evaluate their value.
Although the issue of causality has had less of a role in psychological research in recent years, it remains a defining concern of psychology – and is less typical of some related fields. The basic question involved in causality is the question of whether a par-ticular variable or set of variables causes or brings about a parpar-ticular effect. Many would argue, though this is controversial, that the best and most appropriate way of testing causal propositions is by conducting ‘true’ experiments in which participants have been randomly assigned to conditions which reflect the manipulation of possible causal vari-ables. The archetypal true experiment is the conventional laboratory experiment. Even then, there is considerable room for doubt as to what variable has been manipulated in a true experiment. It is important to check out the possibility that the experimental manipulation has not created effects quite different from the ones that were intended.
Alternative interpretations of the findings should always be a concern of psychologists.
However, the biggest problem is that there are many variables which simply cannot be manipulated by the researcher: for example, it is not possible to manipulate variables such as schizophrenia, gender, social economic status or intelligence for the convenience of testing ideas using true experiments. However, the variety and stimulation of using
the more naturalistic or realistic research methods which are often the only rational choice in field settings is a challenge which many psychologists find rewarding.
Often these are described as non-experimental designs which, from some points of view, might be regarded as a somewhat pejorative term. It is a bit like describing women as non-men. It implies that the randomised experiment is the right and proper way of doing psychology. The truth is that there is no right and proper way of intellectual
Often these are described as non-experimental designs which, from some points of view, might be regarded as a somewhat pejorative term. It is a bit like describing women as non-men. It implies that the randomised experiment is the right and proper way of doing psychology. The truth is that there is no right and proper way of intellectual