Specifications for this HLM2 run

(1)

One way ANOVA model

1. How much do U.S. high schools vary in their mean mathematics achievement? 2. What is the reliability of each school’s sample mean as an estimate of its true

population mean?

3. Do schools vary significantly from each other?

Module: HLM2 (7.00) Date: Jun 15, 2011 Time: 05:47:00

Specifications for this HLM2 run

Problem Title: One Way ANOVA

The data source for this run = HSB.mdm

The command file for this run = C:\Documents and

Settings\manningma\Dropbox\Teaching\HLM Workshop 2011\HSB\One Way ANOVA.hlm Output file name = One Way ANOVA.html

The maximum number of level-1 units = 7185 The maximum number of level-2 units = 160 The maximum number of iterations = 1000

Method of estimation: full maximum likelihood

The outcome variable is MATHACH

Summary of the model specified

Level-1 Model MATHACHij = β0j + rij Level-2 Model β0j = γ00 + u0j Mixed Model MATHACHij = γ00 + u0j+ rij

(2)

Iterations stopped due to small change in likelihood function σ2 = 39.14838 Standard error of σ2 = 0.66054 τ INTRCPT1,β0 8.55379 Standard error of τ INTRCPT1,β0 1.06124

Random level-1 coefficient Reliability estimate INTRCPT1,β0 0.901

The value of the log-likelihood function at iteration 4 = -2.355699E+004

Final estimation of fixed effects:

Fixed Effect Coefficient Standard

error t-ratio

Approx.

d.f. p-value

For INTRCPT1, β0

INTRCPT2, γ00 12.637067 0.243638 51.868 159 <0.001

γ00 : Mean math ach (nothing is predicting it at either level) and associated SE.

You can use this to get a mean and confidence interval around that mean.

Final estimation of fixed effects (with robust standard errors)

error t-ratio

Approx.

d.f. p-value

For INTRCPT1, β0

INTRCPT2, γ00 12.637067 0.243617 51.873 159 <0.001

Final estimation of variance components

Random Effect Standard Deviation Variance Component d.f. χ 2 _p_-value INTRCPT1, u0 2.92469 8.55379 159 1660.22552 <0.001 level-1, r 6.25687 39.14838

(3)

u0: Variance in mean MATHACH between schools. The variance between schools is significant,

χ2

(159) = 1660.23. Schools vary significantly from each other.

rij: Variance in means within school.

The total variance around mean math ach is level-1 (within-school) variance + level-2 (between-school) variance. The Intra-class Correlation Coefficient (ICC) is the proportion of the total variance in math ach that is between groups.

This is exactly the same as

For the current data, ICC = 8.55/(8.55+39.15) = 0.18. Eighteen percent of the variance in math achievement is between-schools.

Statistics for the current model

Deviance = 47113.972342

(4)

Means as outcome model

Predicting the school mean from school SES (MEANSES)

1. Do schools with high MEAN SES also have high math achievement?

2. What amount of between-school variance in math achievement is accounted for by the model with MEAN SES?

3. Do school achievement means vary significantly once MEAN SES is controlled?

Specifications for this HLM2 run

Problem Title: Means as Outcome

Settings\manningma\Dropbox\Teaching\HLM Workshop 2011\HSB\Means as Outcome.hlm Output file name = Means as Outcome.html

The maximum number of level-1 units = 7185 The maximum number of level-2 units = 160 The maximum number of iterations = 1000

Summary of the model specified

Level-1 Model MATHACHij = β0j + rij Level-2 Model β0j = γ00 + γ01*(MEANSESj) + u0j Mixed Model MATHACHij = γ00 + γ01*MEANSESj + u0j+ rij

(5)

Iterations stopped due to small change in likelihood function σ2 = 39.15731 Standard error of σ2 = 0.66065 τ INTRCPT1,β0 2.59327 Standard error of τ INTRCPT1,β0 0.39249

error t-ratio Approx. d.f. p-value For INTRCPT1, β0 INTRCPT2, γ00 12.649740 0.148322 85.286 158 <0.001 MEANSES, γ01 5.862924 0.359134 16.325 158 <0.001

γ00 : Mean Math ach when MEANSES = 0.

γ01: Effect on Math Ach of a one unit change in MEANSES

Random Effect Standard Deviation

Variance

Component d.f. χ

2 _p_-value

(6)

level-1, r 6.25758 39.15731

Note the reduction in between school variance, u0, now that we are accounting for some of it

with MEANSES.

At the between-school level, β0, or mean MATHACH, is equal to some initial value plus an

effect of MEANSES, or in equation form:

β0 = γ00 + γ01MeanSES + u0

We are accounting for some of the between-school unknown variance, u0, with MEANSES. To

calculate just how much of that unknown variance we are accounting for, we calculate the proportion by which the initial variance was reduced when we added MEANSES.

Initial error/unknown variance from previous model = 8.55 Unknown variance when MEANSES is in the model = 2.59 Reduction in variance = 8.55 – 2.59 = 5.96

Proportion reduction in variance = 5.96/8.55 = .697

MEANSES accounted for 69.7% of the variance in mean MATHACH (β0)

It is important to be clear and recall that 18% of the TOTAL variance was at the between-school level (see calculation of ICC above). In other words, recall that u0 accounted for 18% of the total

variance. So we are accounting for 69.7% of that 18% with MEANSES.

Deviance = 46957.270646

Number of estimated parameters = 4

Model comparison test

χ2 statistic = 156.70170 Degrees of freedom = 1 p-value = <0.001

The model comparison test (difference between deviance in current model and previous model) indicates that this is a better fitting model than the previous one.

(7)

Contextual Effects Model (Grand Mean Centered)

Specifications for this HLM2 run

Problem Title: Contextual Effects Model (Grand Mean)

Settings\manningma\Dropbox\Teaching\HLM Workshop 2011\HSB\Contextual Effects Model (Grand Mean).hlm

Output file name = Contextual Effects Model (Grand Mean).html The maximum number of level-1 units = 7185

The maximum number of level-2 units = 160 The maximum number of iterations = 1000

Summary of the model specified

Level-1 Model

MATHACHij = β0j + β1j*(SESij) + rij

Level-2 Model

β0j = γ00 + γ01*(MEANSESj) + u0j

β1j = γ10

Note that we “turned off” the level-2 variance in the SES effect (the Slope). This is saying that the slope, or the effect of SES, is the same across all the schools. In other words, no matter what school you are in, a one unit change in individuals’ SES is equal to a 2.19 unit change in

MATHACH (see fixed effects)

SES has been centered around the grand mean.

SES is centered around the mean for the entire sample of data; this means that it is zero at the mean for the entire sample. So the β0jis now the mean math ach at mean individual SES for the

(8)

MEANSES has been centered around the grand mean.

Likewise, MEANSES (each school’s average SES) has been centered around the mean for the sample. So β0jis mean math ach at the mean for schools’ SES.

Mixed Model

MATHACHij = γ00 + γ01*MEANSESj

+ γ10*SESij + u0j+ rij

You can see it more clearly in the mixed model equation. β0jwhich is γ00 = mean Math Ach

when MEANSES and SES are equal to 0.

Final Results - Iteration 6

Iterations stopped due to small change in likelihood function

σ2 = 37.01402 Standard error of σ2 = 0.62449 τ INTRCPT1,β0 2.64700 Standard error of τ INTRCPT1,β0 0.39306

For SES slope, β1

INTRCPT2, γ10 2.191165 0.108660 20.165 7024 <0.001

(9)

Let’s take a look at the model.

MATHACH = γ00 + γ01MEANSES + γ10SES + u0+ r

The effect of SES on math achievement is defined by two sources. A source at the individual level meant to represent his or her own individual circumstances (SES: family income,

occupation, etc.), and a source at the school level meant to represent the resources of a particular school (MEANSES: mean of SES variable in each particular school). This SES effect is

captured jointly by two coefficients in the model: γ01 andγ10.

Despite that there are two coefficients in the model that capture the effect of SES, there are actually three effects to consider. There is the effect of the moving from a school with one SES to a school with a different SES, which is βb. But this effect is actually composed of the effect of

individuals’ SES on math achievement (βw) and the effect of schools, on the contexts’, effects on

math achievement (βc).

The coefficient for the variable at the individual level (γ10), SES, is always going to be the βw

effect. When we grand mean center SES, what we capture with γ01 is the effect of schools’ mean

SES on math achievement, controlling for the effect of individual SES. This coefficient γ01

quantifies the effect ofbeing in a particular context, namely a particularschool with a particular mean SES. Thus, γ01IS the contextual effect when the individual level component of that effect

is grand-mean centered. Together, both of these coefficients tell us about the effect of SES on math achievement in schools (the cumulative βb effect.) Sometimes, in using a grand mean

centered approach, it can be a bit more intuitive to think about the effects of context controlling for the effects of individual.

Finally, let’s look exactly at what these coefficients tell us. If Kid A and Kid B are in the same school, and Kid A’s SES is greater than Kid B’s SES by one unit, then Kid A’s MATHACH is greater than Kid B’s MATHACH by 2.19 (γ10).

If Kid A and Kid B have the same individual SES, but Kid A is in a different school that is greater by one unit of MEANSES, then Kid A’s MATHACH is 3.67 (γ01) greater than Kid B’s

MATHACH.

For SES slope, β1

(10)

Deviance = 46561.966966

Number of estimated parameters = 5

Contextual Effects Model (Group Mean Centered)

Specifications for this HLM2 run

Problem Title: Contextual Effects Model (Group Mean)

Settings\manningma\Dropbox\Teaching\HLM Workshop 2011\HSB\Contextual Effects Model (Group Mean).hlm

Output file name = Contextual Effects Model (Group Mean).html The maximum number of level-1 units = 7185

The maximum number of level-2 units = 160 The maximum number of iterations = 1000

Summary of the model specified

Level-1 Model

(11)

Level-2 Model

β0j = γ00 + γ01*(MEANSESj) + u0j

β1j = γ10

SES has been centered around the group mean.

SES has been centered around the mean for each school. So SES is equal to 0 at the mean for each school. It might not be the same value for each school, but in the model it has the same

meaning for each school.

MEANSES has been centered around the grand mean.

Mixed Model

MATHACHij = γ00 + γ01*MEANSESj

+ γ10*SESij + u0j+ rij

Final Results - Iteration 6

Iterations stopped due to small change in likelihood function

σ2 = 37.01402 Standard error of σ2 = 0.62449 τ INTRCPT1,β0 2.64694 Standard error of τ INTRCPT1,β0 0.39306

For SES slope, β1

(12)

The contextual effect with group-mean centering of level-1 variable:

The model is exactly the same:

MATHACH = γ00 + γ01MEANSES + γ10SES + u0+ r

The only thing that we have changed is how we choose to center the within-unit variable, SES. We have group mean centered it.

When we group mean center, coefficient γ01 is now the cumulative βb effect. So to get the effect

of context, we subtract the individual level effect, γ10, fromγ01. This underscores the fact that you

have to be mindful of your choice of centering when it comes to introducing effects that exist at both the aggregate and the individual levels, and further underscores the care needed in

interpreting your coefficients in these kinds of models.

For SES slope, β1

INTRCPT2, γ10 2.191172 0.129367 16.938 7024 <0.001

Deviance = 46561.964487