- After designing an experiment, you must think of the best way to conduct the experiment - Choosing participants, what instructions to give, materials to use
Reliability of Measurements
- Reliability: repeatability of a measurement, the likelihood that if the measurement was made again, it would yield the same value
if we say a procedure has high reliability, it means that it produces consistent results under consistent conditions
ex. measuring a person’s height is reliable academic aptitude is reliable but a little less - Reliable ≠ valid
- When a person blinks is reliable, but if we define the detection of the 3D image as the time it takes until the person blinks, this is not valid
- Many things could have caused the person to blink, not just the detection of the 3D image - Achieving reliability is easier than achieving validity
External Factors Decreasing Reliability
- Sometimes there are external factors that can decrease the reliability of a variable
- Ex. some images projected were not properly scanned and so were out of focus, this will cause differences in measurement of detection among participants
- There are some things that could help with the control of extraneous factors that affect reliability of measurements
- Give the same set of instructions to each participant - Make sure all equipment functions properly
- Assistants are well trained
- Noise and other distractions are kept to a minimum Subjectivity and Reliability
- Subjectivity is a factor that could affect reliability
- At times, some measurements are subjective and requires judgment and expertise
- Ex. we want to study the number of friendly interactions that a child has with other children in a group
but a “friendly interaction” to me may not be friendly to you - One possible solution is to specify a friendly interaction
- Another solution is to have more than one observer and have them score independently - If their ratings agree, the scoring system has a high interrater reliability
- Interrater reliability: the degree to which two or more independent observers agree in their ratings of another organism’s behavior
- If their ratings disagree, the interrater reliability is low and the experiment has no point in continuing
- That’s why it’s important that a rating system is defined well and that raters are trained properly Selecting the Participants
- Who should we choose? How do we split the participants into the control and experimental groups?
- If we don’t randomly assign participants, conclusions will not be valid
- Ex. research on which teaching method is better, one taught at 8 am or one taught at 4 pm (students choose which class to sign up for)
there may be too many differences between the two groups of participants
maybe the 8 am students are athletes who have practices in the afternoon
maybe the 4 pm students all like to sleep in
- Random assignment: procedure in which each participant has an equally likely chance of being assigned to any of the conditions/groups of an experiment
a common way to avoid confounding participant characteristics with the values of the independent variable
you could toss a coin
people have different abilities, personalities, etc. that could affect outcome of the experiment, but if we do random assigning, then the differences are equally spread across the groups
Anger Experiment
- They wanted to study whether anger has an effect on ability to concentrate - Experimental group: were treated rudely by experimenters
- Control group: were treated politely
- Both groups were asked to identify when a certain letter appeared
- Problem: some angry participants of the experimental group walked out of the experiment - The control group and the leftover of the experiment group are no longer equivalent, they have
distinguishable differences in personality
- Experimental group: people who will put up with rudeness
- Control group: people who will or will not put up with rudeness
- Moral: sometimes confounded variables occur while the experiment is under way - There is no solution to this particular problem
Expectancy Effects
- The Hawthorne effect: when participants in an experiment know that they’re being observed, it may affect their behavior
observation can change that which you observe
- Ex. experiment where they tested whether increasing the level of lighting in the plant would increase productivity of workers
results: yes, but it was short-lived
people’s productivity actually increased even when they decreased the light levels
explanation: the workers knew that they were being observed on their productivity - Eventually, methods were developed where the Hawthorne effect was countered - Sometimes, participants try to help the researchers confirm their hypothesis
there is a type of cooperation where participants, knowing the hypothesis, will sometimes unintentionally behave in the way to make the hypothesis true
- That is why researchers at times don’t disclose their hypothesis until after the dependent variable has been measured
- Rarely, researchers may use deception, providing the participant with an alternative explanation for the experimental events to prevent the person from purposely confirming the hypothesis - However, when using deception, researchers must tell participants the truth as soon as they could
to regain their trust
Techniques to Cope with Hypothesis Awareness Single-Blind Experiments
- Single-blind study: experiment in which the researcher but not the participant knows the value of the independent variable
- Ex. we want to study whether a stimulant drug has any effect on a person’s ability to perform a task requiring fine manual dexterity
- We could get one group to take the pill and the other not to, and then see how many needle threading are done in a 10 minute period
- Problem: the administration of a drug itself have an effect on behavior, now you have two independent variables, on top fo the physiological effects of the drugs, you have the administration of the drug
- To cope, we can do a single-blind study
- Both groups take a pill, but they don’t know if it’s a stimulant drug or a placebo
- Placebo: inert substance that can’t be distinguished in appearance from a real medication; it is used as the control substance in a single-blind/double-blind experiment
- Participants now only know that they have 50-50 chance that they took the stimulant drug
Double-Blind Experiments
- Double-blind study: experiment in which neither participant, nor researcher knows the value of the independent variable
often used when observation is subjective for the researcher
- Ex. we want to test whether a psychological disorder person taking a drug would cause them to be more willing to engage in conversation (because enhanced communicability would facilitate their therapy)
- Their “quality of conversation” is a difficult dependent variable to measure and the rating is subjective, which is why researchers also should not know whether the participant received the drug or the placebo
- If the researcher doesn’t know, then the ratings of the conversation quality won’t be affected by any preconceived ideas
- Ex. we want to see if a type of psychotherapy causes a person to be more willing to engage in conversation
- The person doing the psychotherapy and the person rating the conversation should be a different person
Correlational Studies
- There are some things that a researcher cannot manipulate, for example, a person’s sex, genetic history, income, social class, family environment, personality
- But these factors may affect behavior - A correlational study studies these factors
- Correlational study: the examination of 2 or more measurements of behavior or other characteristics of people/animals
in a correlational study, we measure 2 variables and determine if they are related, using the statistical procedure correlation
Shyness Experiment
- If two variables are correlated, we cannot necessarily say that there is cause and effect relation - There was a study that hypothesized that shy people tend to daydream more than less shy people - They looked at the relation between the shyness of a person and the time per day they septn
daydreaming
- Results showed that shy people spent more time daydreaming
- That gives us a correlation, it shows that shyness and daydreaming are related - However, we cannot make a cause and effect conclusion
- We cannot tell whether shyness causes daydreaming or vice versa, or another variable that causes shyness and more daydreaming
Major in University Experiment
- Allen conducted a study that was focused on the employability of social science graduates
- He tracked the income growth of graduates with bachelor’s degree in social sciences from their early 20s to their 50s (the peak)
- He also did the same for graduates of other educational programs
- Results showed that the income growth was the highest for social science graduates
- There is a correlation between graduating from a social science program and long-term career growth
- This correlation doesn’t imply a cause and effect relation, there are many other factors - It could be that people who have a secure career path take social sciences
- It could be that people who take social sciences have good people skills which brings them success in their positions
- To determine if there is a causal role, we would have to randomly assign participants to various programs, then track their income (do an experiment)
- Since we cannot make such an interference, we can only accept the correlation as suggestive Another Correlation Example
- People who read a certain newspaper have a high income - This is a correlation, not necessarily a cause and effect relation
- This doesn’t mean that reading the newspaper would increase your income
- It could be that people with high incomes reach the newspaper because there is news about their profession
How to Reduce Uncertainty in Correlational Studies
- Matching: a systematic selection of participants in groups in an experiment/correlational study to ensure that the mean values of important participant variables of the groups are similar
instead of selecting participants randomly, we match the participants in each of the groups on all of the relevant variables expect the one being studied
- Ex. for the shyness experiment, we can separate 2 groups: shy and not shy, then make sure that both groups have the same average age, intelligence, income, personality
- If we still find a relation between shyness and increased daydreaming, then we can say that there is in fact a relation of cause and effect between the two variables (we now have no 3rd variable to worry about)
- Limitation to the matching procedure: we may not know all the variables that should be held constant
if the two groups are not matched on an unknown important variable, results will be misleading
- Also, even with the matching procedure, we do not know which variable is the cause and which is the effect