Definitions:

Chapter 13

Experimental Design

What I will know and be able to do

Use the four principles of experiment design to design a

completely randomized experiment and compare

responses of different groups.

Assignment:

Read Chapter 13!!

Definitions:

1)

Observational study

- observe outcomes

without imposing any treatment

I’ve developed a new rabbit food, Hippity Hop.

Rabbit Food

Makes fur

soft &

shiny!

Increases

energy!

100% of daily

vitamins &

Can I just make these claims?

What must I do to make these claims?

Who (what) should I test this on?

What do I test?

NO

Do an experiment

Rabbits

3)

Experimental unit

– the single individual

(person, animal, plant, etc.) to which the

different treatments are assigned

4)

Factor

– is the explanatory variable

6)

Response variable

– what you measure

7)

Treatment

– a specific experimental condition

I plan to test my new rabbit food.

What are my experimental units?

What is my factor?

What is the response variable?

Rabbits

Type of food

Hippity Hop

I’ll use my pet

rabbit, Lucky!

Since Lucky’s coat is shinier & he has

8)

Control group

– a group that is used to compare

the factor against; can be a placebo or the “old” or

current item

9)

Placebo

– a “dummy” treatment that can have no

Old Food Hippity Hop

Now I’ll use Lucky & my

friend’s rabbit, Flash.

Lucky gets Hippity Hop

food & Flash gets the

old rabbit food.

Old Food _{Hippity Hop}

The first five rabbits

that I catch will get

Hippity Hop food and

the remaining five will

get the old food.

Old Food _{Hippity Hop}

Number the rabbits from 1 –

10.

Place the numbers in a hat.

1 2 3 4 5 6 7 8 9 ₁₀

The first five numbers pulled

from the hat will be the

rabbits that get Hippity Hop

food.

I

evaluated the rabbits & found that the rabbits

eating Hippity Hop are better than the old food!

The remaining rabbits get

the old food.

10)

blinding

- method used so that units do not

know which treatment they are getting

11)

double blind

- neither the units nor the

Rabbit Food

Hippity Hop

Rabbit Food

makes fur soft

and shiny, &

increases

energy for

ALL

types of

rabbits!

Principles of Experimental Design

•

Control

of effects of extraneous variables on the

response – by comparing treatment groups to a control

group (placebo or “old”)

•

Replication

of the experiment on many subjects to

quantify the natural variation in the experiment

•

Randomization

–

the use of chance to assign subjects

The ONLY

way to show cause

& effect is with a

well-designed,

Example 1:

A farm-product manufacturer wants to determine if the

yield of a crop is different when the soil is treated with three different

types of fertilizers. Fifteen similar plots of land are planted with the

same type of seed but are fertilized differently. At the end of the

growing season, the mean yield from the sample plots is compared.

Experimental units?

Factors?

Levels?

Response variable?

How many treatments?

Plots of land

Type of fertilizer

Fertilizer types A, B, & C

Yield of crop

Example 2

: A consumer group wants to test cake pans to

see which works the best (bakes evenly). It will test

aluminum, glass, and plastic pans in both gas and electric

ovens.

Experiment units?

Factors?

Levels?

Response variable?

Number of treatments?

Two factors - type of pan & type of oven

Type of pan has 3 levels (aluminum, glass, & plastic & type of oven has 2 levels (electric & gas)

How evenly the cake bakes

6

Example 3

:

A farm-product manufacturer wants to determine if

the yield of a crop is different when the soil is treated with three

different types of fertilizers. Fifteen similar plots of land are planted

with the same type of seed but are fertilized differently. At the end

of the growing season, the mean yield from the sample plots is

compared.

Why is the same type of seed used on all 15 plots?

What are other potential extraneous variables?

Does this experiment have a placebo? Explain

It is part of the controls in the experiment.

Type of soil, amount of water, etc.

Experiment Designs

•

Completely randomized – all experimental units are

allocated at random among all treatments

Treatment group 1 Treatment group 2 Treatment group 3 explanatory

variable

Treatment C

Treatment B

Completely randomized design

Randomly assign

experimental units to

treatments

Treatment A

•

Randomized block

– units are blocked into groups

(homogeneous) and then randomly assigned to

treatments

Treatment B

Randomized block design

Randomly assign

experimental units to

treatments

Treatment A

Put into homogeneous

groups

•

match up experimental units according to similar

characteristics & randomly assign one to one treatment &

the other automatically gets the 2nd treatment

•

have each unit do both treatments in random order

•

the assignment of treatments is

dependent

Matched pairs

–

Pair experimental

units according to

specific

characteristics.

Next, randomly assign

one unit from a pair to

Treatment A. The

other unit gets

Treatment B.

Treatment A Treatment B

This is one way to do a matched

pairs design – another way is to have

the individual unit do both

12)

Confounding variable

– the effect of the

confounding variable on the response cannot

be separated from the effects of the

Suppose we wish to test a new deodorant

against one currently on the market.

• Ask for 4 male & 4 female volunteers

• Randomly assign to treatments – no confounding between

gender & deodorant

• Block by gender & randomly assign – no confounding

• Block by gender – give females new deodorant & males get

current – NOW have

Treatment B

Treatment & group are confounded

Treatment A Treatment B

One group is

assigned to

treatment A & the

other group to

treatment B.

Confounding

does

NOT

Example 5:

An article from

USA Today

reports the number of

victims of violent crimes per 1000 people. 51 victims have

never been married, 42 are divorced or separated, 13 are

married, and 8 are widowed.

Is this an experiment? Why or why not?

What is a potential confounding variable?

Age – younger people are more at risk to be victims of violent crimes

Example 6:

Four new word-processing programs are to be

compared by measuring the speed with which standard tasks can be

completed. One hundred volunteers are randomly assigned to one of

the four programs and their speeds are measured.

Is this an experiment? Why or why not?

What type of design is this?

Factors? Levels?

Response variable?

Yes, a treatment is imposed. Completely randomized one factor: word-processing program with 4 levels

Example 6

: Four new word-processing programs are to be

compared by measuring the speed with which standard tasks

can be completed. One hundred volunteers are randomly

designed to one of the four programs and their speeds are

measured.

Is there a potential confounding variable?

Can this design be improved?

Explain.

NO, completely randomized designs have no confounding

You could do a block

design where each person uses each program in

Example 7

:

Suppose that the manufacturer wants to test a new fertilizer

against the current one on the market. Ten 2-acre plots of land scattered

throughout the county are used. Each plot is subdivided into two subplots,

one of which is treated with the current fertilizer, and the other with the

new fertilizer. Wheat is planted and the crop yields are measured.

What type of design is this? Why

use this method?

When does randomization occur?

Matched - pairs design

Randomization

reduces bias by spreading any

uncontrolled confounding variables evenly throughout the

treatment groups.

Variability

is controlled by sample size. Larger

samples produce statistics with less variability.

Blocking

also helps reduce variability.

Is there another way

to reduce variability?

High bias & low variability

Low bias & low variability Low bias & high variability

Designing an experiment

The 4 Principles of Experimental Design

• Control: Make sure that all other conditions (besides experimental factors) are the same for all subjects (to isolate the effects)

• Randomize: Treatments are randomly assigned to subjects so that the

experiment does not favor one group over the other.

• Replicate: The experiment should be repeated on a large number of

subjects (or in different groups within the population)

• Block: Use to reduce the effects of identifiable attributes of the subjects that cannot be controlled (variation) – this recognizes differences

Designing an experiment

• State what you want to know. (Know whether Hippity Hop makes rabbits healthier…”)

• Specify the response variable. (Evaluate the softness of fur and weight of each rabbit)

• Specify the factors, the levels and the treatments (Factor is the food, 2 levels – original and Hippity Hop, which means there are 2 treatments)

• Specify the experimental units (Rabbits)

• Observe the principles of the design:

• Control any sources of variability that you can

• Randomly assign experimental units to treatments – block if necessary

• Replicate the results

Definition

Statistically Significant

When an observed difference is too large for us to

believe that it is likely to have occurred naturally,

we consider the difference to be statistically

significant.