Introduction to Analysis of Variance (ANOVA) Limitations of the t-test

Introduction to Analysis of Variance (ANOVA)

The Structural Model, The Summary Table, and the One-

Way ANOVA

Limitations of the t-Test

• Although the t-Test is commonly used, it has limitations

– Can only test differences between 2 groups

• High school class? College year?

– Can examine ONLY the effects of 1 IV on 1 DV – Limited to single group OR repeated measures

designs

Limitations of the t-Test

• Testing differences between group means

– IV: Gender (Male & Female)

– IV: High-school class (First-year, Sophomore, Junior, & Senior)

– Using the t-Test, we must either “collapse”

categories… or not run the analysis

Limitations of the t-Test

• 1 Independent Variable

– Gender differences in depression

• IV: Gender (Male & Female)

• DV: Level of depression (BDI score)

• 2 Independent Variables

– Gender and social support on depression

• IV₁: Gender (Male & Female)

• IV₂: Social support (High, Medium, & Low)

• DV: Level of depression (BDI score)

Limitations of the t-Test

• 2 or more Independent Variables

– Simultaneously examine the impact of 2 or more IVs on a single DV

– Examine how the effects of 2 or more IVs COMBINE to affect a single DV

Limitations of the t-Test

• Single time point OR repeated measures designs

– 1 group at 2 time points = repeated measures – 2 groups at 1 time point = independent groups

• Single time point AND repeated measures designs

– 2 or more groups at 2 or more time points

The Analysis of Variance (ANOVA)

• The ANOVA can test hypotheses that the t-Test cannot

• Probably the most commonly abused statistical test

• Many varieties of ANOVA

– One-Way (between subjects)

– Factorial ANOVA (between or within subjects) – Repeated Measures (within subjects)

– Mixed-Model (between & within subjects)

Varieties of ANOVA

• One-Way ANOVA

– 1 continuous Dependent Variable

– 1 Independent Variable consisting of 2 or more “categorical” groups

• The one-way ANOVA with 2 groups is “equivalent”

to the independent groups t-Test

Varieties of ANOVA

• Factorial ANOVA

– 1 continuous Dependent Variable

– 2 or more Independent Variables consisting of 2 or more “categorical” groups for each IV

• 2 IVs = Two-Way Factorial ANOVA

• 3 IVs = Three-Way Factorial ANOVA

– We call these “factorial” designs because EACH level of each IV is paired with EVERY level of ALL other IVs

2 x 2 Contingency Table

Col 2Total Col 1 Total

Row 2 Total

DATA DATA

Level 2

Row 1 Total

DATA

Level 1 DATA

IV 1

Level 2 Level 1

IV 2

Note: Each Level 1 of IV 1 is paired with BOTH Level 1 and Level 2 of IV 2

2 x 2 Contingency Table

Col 2Total Col 1 Total

Row 2 Total

DATA

Female DATA

Row 1 Total

DATA

Male DATA

Gender

Low High

Social Support

Note: Each Level 1 of IV 1 is paired with BOTH Level 1 and Level 2 of IV 2

Varieties of ANOVA

• Repeated Measures ANOVA

– Time points = IV

– The DV is assessed at EACH time point

Varieties of ANOVA

• Mixed-Model ANOVA

– 1 continuous Dependent Variable

– 1 or more Independent Variables consisting of 2 or more “categorical” groups (between)

– 1 Independent Variable consisting of 2 or more “categorical” time points (within)

• The DV is assessed at EACH time point

ANOVA

• ANOVA models we will consider

– One-Way ANOVA

– Two- and Three-way Factorial ANOVA – Repeated measures ANOVA

– Mixed-model ANOVA

Choosing the Best Test

The Underlying Model

• A statistical model by example:

• Assume: the average 18 year old human being weighs approximately 138 pounds

– Men, on average, weigh 12 pounds more than the average human weight

– Women, on average, weigh 10 pounds less than the average human weight

The Underlying Model

• For any given human being, I can break weight down into 3 components:

– Average weight for all individual

• 138 lbs

– Average weight for each group

• Men: +12 lbs

• Women: - 10 lbs

– The individual’s unique difference

The Underlying Model

• Male weight

Weight = 138 lbs + 12 lbs + uniqueness

• Female weight

Weight = 138 lbs – 10 lbs + uniqueness

• If you understand this process, you

understand the basic theory behind the

ANOVA

Partitioning Variance

• The idea behind the ANOVA test is to divide or separate (partition) variance observed in the data into categories of what we CAN and what we CANNOT explain

Total Variance

Explained Unexplained

The Structural Model

• Mathematically, we partition the total variance of our data using the structural form of the ANOVA model

– X_ij= µ + τ_j +ε_ij

– The structural model translates as follows:

The score for any single individual is equal to the sum of the population mean plus the

mean of the group plus the individual’s unique contribution

The Structural Model

• For our weight example:

– µ = population weight = 138 lbs

– τ = group difference in weight = 12 or 10 lbs – ε = “unique” contribution of an individual’s

score

– µ & τ can be explained – ε cannot be explained…

Uniqueness

• Oftentimes, we value our uniqueness…

– In statistics, unique variance is BAD

– Since we can’t explain unique variance, we call it “error”

– Thus, the ANOVA seeks to examine the relative proportion of explainable variance in our data to the unexplainable variance

Assumptions of the ANOVA

• Owing to the mathematical construction of the ANOVA, the underlying assumptions of the test are very important

– Homogeneity of variance – Normality

– Independence of Observations – The Null Hypothesis

Homogeneity of Variance

• Homogeneity of variance refers to the variance for each group being equal to the variance of every other group

– Really, we mean that the variance of each group is equal to the variance of the error for the total analysis

– σ₁² = σ₂² = σ₃² = σ_j² = σ_e²

Homogeneity of Variance

• Heterogeneity of variance is another BAD thing

– Heterogeneous variances can greatly influence the results you obtain, making it either more or less likely that you will reject H₀ – Visual inspection of variances

– Tests of homogeneity of variance

Normality

• The ANOVA procedure assumes that scores are normally distributed

– More accurately, it assumes that ERRORS are normally distributed

– Random sampling and random assignment – Lacking normality, consider mathematical

transformations

• Logarithmic & square root transformations

Independence of Observations

• Simple: The scores for 1 group are not dependent on the scores from another group

– Don’t share subjects between groups – If violated…

• What is wrong with your experimental design?

• Are you using the appropriate test?

The Null Hypothesis

• Less an assumption and more a theoretical point:

– H₀: µ₁ = µ₂ = µ₃ = µ₄= µ₅

• This is almost ALWAYS the basic form of

your null hypothesis…

Calculating the One-Way ANOVA

• In order to calculate the One-Way ANOVA statistic, we need to complete a number of intermediate steps

• Because there are several intermediate steps, we keep track of our progress with something called a summary table

The Summary Table

Total Error Treatment

F Mean Square

(MS) Sum of Squares

(SS) df

Source

One-Way ANOVA:

Partitioning Variance

• The idea behind the ANOVA test is to divide or separate (partition) variance observed in the data into categories of what we CAN and what we CANNOT explain

Total Variance

Treatment Error

The Summary Table

(N-1)

Total

k(n-1)

Error

(k-1)

Treatment

F Mean Square

(MS) Sum of Squares

(SS) df

Source

Note: df + df =df

Sum of Squares

• Note:

– = The treatment group mean

– = The grand mean (mean of all scores) – X_ij = Each individual score

xj

x..

The Summary Table

(N-1)

Total

k(n-1)

Error

(k-1)

Treatment

F Mean Square

(MS) Sum of Squares

(SS) df

Source

2 ..) (X X n

SS_Treatment ⁼∑ _j⁻

Treatment Total

Error SS SS

SS = −

2 ..) (X X

SS_Total ⁼

∑

The Summary Table

(N-1)

Total

k(n-1)

Error

(k-1)

Treatment

F Mean Square

(MS) Sum of Squares

(SS) df

Source

2 ..) (X X n

SS_Treatment ⁼∑ _j⁻

Treatment Total

Error SS SS

SS = −

2 ..) (X X

SS_Total ⁼

∑

Treatment Treatment Treatment

df MS =SS

Error Error Error

df MS =SS

The Summary Table

(N-1)

Total

k(n-1)

Error

(k-1)

Treatment

F Mean Square

(MS) Sum of Squares

(SS) df

Source

2 ..) (X X n

SS_Treatment⁼∑ _j⁻

Treatment Total

Error SS SS

SS = −

2 ..) (X X

SS_Total ⁼

∑

Treatment Treatment Treatment

df MS =SS

Error Error Error

df MS = SS

Error Treatment

MS F=MS

Example:

Anorexia Nervosa 3 Group Tx

2 3 6 -4

0 -1.5

-10

2 4

-3

9 2

1

6 -1

-8

3.5 0

-5

3 0

2

4 2

-2

0 -2

-1.5

4 -1

0

6 1

-6

4 0

-5

CBT IPT

Control

Example:

Anorexia Nervosa 3 Group Tx

Descriptives Change in Weight

12 -3.4583 3.60214 1.03985 -5.7470 -1.1696

11 .3182 1.79266 .54051 -.8861 1.5225

14 3.7500 2.45537 .65623 2.3323 5.1677

37 .3919 4.04512 .66501 -.9568 1.7406

Control IPT CBT Total

N Mean Std. Deviation Std. Error Lower Bound Upper Bound 95% Confidence Interval for

Mean

Example:

Anorexia Nervosa 3 Group Tx

Test of Homogeneity of Variances Change in Weight

2.666 2 34 .084

Levene Statistic df1 df2 Sig.

H₀: σ₁ = σ₂= σ₃ H₁: σ₁ ≠ σ₂≠ σ₃ Fail to reject H_{0 . . .}

Example:

Anorexia Nervosa 3 Group Tx

ANOVA Change in Weight

335.827 2 167.914 22.544 .000

253.241 34 7.448

589.068 36

Between Groups Within Groups Total

Sum of Squares df Mean Square F Sig.

F(2,34) = 22.54, p < .05

Example:

Anorexia Nervosa 2 Group Tx

2 3 6 -4

0 -10

2 -3

9 1

6 -8

3.5 -5

3 2

4 -2

0 -1.5

4 0

6 -6

4 -5

CBT Control

Example:

Anorexia Nervosa 2 Group Tx

Descriptives Change in Weight

12 -3.4583 3.60214 1.03985 -5.7470 -1.1696 -10.00 2.00

14 3.7500 2.45537 .65623 2.3323 5.1677 .00 9.00

26 .4231 4.71952 .92557 -1.4832 2.3293 -10.00 9.00

Control CBT Total

N Mean Std. Deviation Std. Error Lower Bound Upper Bound 95% Confidence Interval for

Mean

Minimum Maximum

Test of Homogeneity of Variances Change in Weight

2.281 1 24 .144

Levene Statistic df1 df2 Sig.

Example:

Anorexia Nervosa 2 Group Tx

H₀: σ₁ = σ₂= σ₃ H₁: σ₁ ≠ σ₂≠ σ₃ Fail to reject H₀

ANOVA Change in Weight

335.742 1 335.742 36.443 .000

221.104 24 9.213

556.846 25

Between Groups Within Groups Total

Sum of Squares df Mean Square F Sig.

Example:

Anorexia Nervosa 2 Group Tx

F(1,24) = 36.44, p < .05

Example:

Anorexia Nervosa 2 Group Tx

• Independent Groups t-test

Group Statistics

12 -3.4583 3.60214 1.03985

14 3.7500 2.45537 .65623

Experimental Group Control CBT Change in Weight

N Mean Std. Deviation Std. Error Mean

• One-way ANOVA

Descriptives Change in Weight

12 -3.4583 3.60214 1.03985 -5.7470 -1.1696

14 3.7500 2.45537 .65623 2.3323 5.1677

26 .4231 4.71952 .92557 -1.4832 2.3293

Control CBT Total

N Mean Std. Deviation Std. Error Lower Bound Upper Bound 95% Confidence Interval for

Mean

Example:

Anorexia Nervosa 2 Group Tx

• Independent groups t-test

Independent Samples Test

2.281 .144 -6.037 24 .000 -7.20833 1.19406

-5.862 18.962 .000 -7.20833 1.22960

Equal variances assumed Equal variances not assumed Change in Weight

F Sig.

Levene's Test for Equality of Variances

t df Sig. (2-tailed) Mean Difference Std. Error Difference t-test for Equality of Means

• One-way ANOVA

ANOVA Change in Weight

335.742 1 335.742 36.443 .000

Between Groups

Sum of Squares df Mean Square F Sig.

Example:

Anorexia Nervosa 2 Group Tx

• Shouldn’t the results of the F- and t-tests be equal if the tests are equivalent?

– F(1,24) = 36.44, p < .05 – t(24) = -6.04, p < .05

• t

= F