**One way ANOVA model **

**1. ** **How much do U.S. high schools vary in their mean mathematics achievement? **
**2. ** **What is the reliability of each school’s sample mean as an estimate of its true **

**population mean? **

**3. ** **Do schools vary significantly from each other? **

Module: HLM2 (7.00) Date: Jun 15, 2011 Time: 05:47:00

**Specifications for this HLM2 run **

Problem Title: One Way ANOVA

The data source for this run = HSB.mdm

The command file for this run = C:\Documents and

Settings\manningma\Dropbox\Teaching\HLM Workshop 2011\HSB\One Way ANOVA.hlm Output file name = One Way ANOVA.html

The maximum number of level-1 units = 7185 The maximum number of level-2 units = 160 The maximum number of iterations = 1000

Method of estimation: full maximum likelihood

The outcome variable is MATHACH

**Summary of the model specified **

**Level-1 Model**

*MATHACHij*=

*β0j*+

*rij*

**Level-2 Model**

*β0j*=

*γ00*+

*u0j*

**Mixed Model**

*MATHACHij*=

*γ00*+

*u0j*+

*rij*

**Iterations stopped due to small change in likelihood function**
σ2
= 39.14838
Standard error of σ2
= 0.66054
τ
INTRCPT1,*β0* 8.55379
Standard error of τ
INTRCPT1,*β0* 1.06124

Random level-1 coefficient Reliability estimate
INTRCPT1,*β0* 0.901

The value of the log-likelihood function at iteration 4 = -2.355699E+004

**Final estimation of fixed effects: **

Fixed Effect Coefficient Standard

error *t*-ratio

Approx.

*d.f.* *p*-value

For INTRCPT1, *β0*

INTRCPT2, *γ00* 12.637067 0.243638 51.868 159 <0.001

*γ00* : Mean math ach (nothing is predicting it at either level) and associated *SE*.

You can use this to get a mean and confidence interval around that mean.

**Final estimation of fixed effects **
**(with robust standard errors) **

Fixed Effect Coefficient Standard

error *t*-ratio

Approx.

*d.f.* *p*-value

For INTRCPT1, *β0*

INTRCPT2, *γ00* 12.637067 0.243617 51.873 159 <0.001

**Final estimation of variance components **

Random Effect Standard
Deviation
Variance
Component *d.f.* χ
2 _{p}_{-value }
INTRCPT1, *u0* 2.92469 8.55379 159 1660.22552 <0.001
level-1, *r* 6.25687 39.14838

*u0: *Variance in mean MATHACH between schools. The variance between schools is significant,

χ2

(159) = 1660.23. Schools vary significantly from each other.

* r*ij: Variance in means within school.

**The total variance around mean math ach is level-1 (within-school) variance + level-2 **
**(between-school) variance. The Intra-class Correlation Coefficient (ICC) is the proportion **
**of the total variance in math ach that is between groups. **

**This is exactly the same as **

**For the current data, ICC = 8.55/(8.55+39.15) = 0.18. Eighteen percent of the variance in **
**math achievement is between-schools. **

**Statistics for the current model **

Deviance = 47113.972342

**Means as outcome model **

**Predicting the school mean from school SES (MEANSES) **

**1. ** **Do schools with high MEAN SES also have high math achievement? **

**2. ** **What amount of between-school variance in math achievement is accounted for by **
**the model with MEAN SES? **

**3. ** **Do school achievement means vary significantly once MEAN SES is controlled? **

Module: HLM2 (7.00) Date: Jun 15, 2011 Time: 05:48:22

**Specifications for this HLM2 run **

Problem Title: Means as Outcome

The data source for this run = HSB.mdm

The command file for this run = C:\Documents and

Settings\manningma\Dropbox\Teaching\HLM Workshop 2011\HSB\Means as Outcome.hlm Output file name = Means as Outcome.html

The maximum number of level-1 units = 7185 The maximum number of level-2 units = 160 The maximum number of iterations = 1000

Method of estimation: full maximum likelihood

The outcome variable is MATHACH

**Summary of the model specified **

**Level-1 Model**

*MATHACHij*=

*β0j*+

*rij*

**Level-2 Model**

*β0j*=

*γ00*+

*γ01**(

*MEANSESj*) +

*u0j*

**Mixed Model**

*MATHACHij*=

*γ00*+

*γ01**

*MEANSESj*+

*u0j*+

*rij*

**Iterations stopped due to small change in likelihood function**
σ2
= 39.15731
Standard error of σ2
= 0.66065
τ
INTRCPT1,*β0* 2.59327
Standard error of τ
INTRCPT1,*β0* 0.39249

Random level-1 coefficient Reliability estimate
INTRCPT1,*β0* 0.737

The value of the log-likelihood function at iteration 6 = -2.347864E+004

**Final estimation of fixed effects: **

Fixed Effect Coefficient Standard

error *t*-ratio
Approx.
*d.f.* *p*-value
For INTRCPT1, *β0*
INTRCPT2, *γ00* 12.649740 0.148322 85.286 158 <0.001
MEANSES, *γ01* 5.862924 0.359134 16.325 158 <0.001

*γ00* : Mean Math ach when MEANSES = 0.

*γ01: *Effect on Math Ach of a one unit change in MEANSES

**Final estimation of fixed effects **
**(with robust standard errors) **

Fixed Effect Coefficient Standard

error *t*-ratio
Approx.
*d.f.* *p*-value
For INTRCPT1, *β0*
INTRCPT2, *γ00* 12.649740 0.148357 85.265 158 <0.001
MEANSES, *γ01* 5.862924 0.320231 18.308 158 <0.001

**Final estimation of variance components **

Random Effect Standard Deviation

Variance

Component *d.f.* χ

2 _{p}_{-value }

level-1, *r* 6.25758 39.15731

Note the reduction in between school variance, *u0*, now that we are accounting for some of it

with MEANSES.

At the between-school level, β0, or mean MATHACH, is equal to some initial value plus an

effect of MEANSES, or in equation form:

β0 = γ00 + γ01MeanSES + u0

We are accounting for some of the between-school unknown variance, *u*0, with MEANSES. To

calculate just how much of that unknown variance we are accounting for, we calculate the proportion by which the initial variance was reduced when we added MEANSES.

Initial error/unknown variance from previous model = 8.55 Unknown variance when MEANSES is in the model = 2.59 Reduction in variance = 8.55 – 2.59 = 5.96

Proportion reduction in variance = 5.96/8.55 = .697

MEANSES accounted for 69.7% of the variance in mean MATHACH (β0)

It is important to be clear and recall that 18% of the TOTAL variance was at the between-school level (see calculation of ICC above). In other words, recall that u0 accounted for 18% of the total

variance. So we are accounting for 69.7% of that 18% with MEANSES.

**Statistics for the current model **

Deviance = 46957.270646

Number of estimated parameters = 4

Model comparison test

χ2 statistic = 156.70170
Degrees of freedom = 1
*p*-value = <0.001

The model comparison test (difference between deviance in current model and previous model) indicates that this is a better fitting model than the previous one.

**Contextual Effects Model (Grand Mean Centered) **

Module: HLM2 (7.00) Date: Jun 15, 2011 Time: 05:50:43

**Specifications for this HLM2 run **

Problem Title: Contextual Effects Model (Grand Mean)

The data source for this run = HSB.mdm

The command file for this run = C:\Documents and

Settings\manningma\Dropbox\Teaching\HLM Workshop 2011\HSB\Contextual Effects Model (Grand Mean).hlm

Output file name = Contextual Effects Model (Grand Mean).html The maximum number of level-1 units = 7185

The maximum number of level-2 units = 160 The maximum number of iterations = 1000

Method of estimation: full maximum likelihood

The outcome variable is MATHACH

**Summary of the model specified **

**Level-1 Model**

*MATHACHij* = *β0j* + *β1j**(*SESij*) + *rij*

**Level-2 Model **

*β0j* = *γ00* + *γ01**(*MEANSESj*) + *u0j*

*β1j* = *γ10*

Note that we “turned off” the level-2 variance in the SES effect (the Slope). This is saying that the slope, or the effect of SES, is the same across all the schools. In other words, no matter what school you are in, a one unit change in individuals’ SES is equal to a 2.19 unit change in

MATHACH (see fixed effects)

SES has been centered around the grand mean.

SES is centered around the mean for the entire sample of data; this means that it is zero at the
mean for the entire sample. So the *β0j*is now the mean math ach at mean individual SES for the

MEANSES has been centered around the grand mean.

Likewise, MEANSES (each school’s average SES) has been centered around the mean for the
sample. So *β0j*is mean math ach at the mean for schools’ SES.

**Mixed Model **

*MATHACHij* = *γ00* + *γ01***MEANSESj*

+ *γ10***SESij* + *u0j*+ *rij *

You can see it more clearly in the mixed model equation. *β0j*which is *γ00* = mean Math Ach

when MEANSES and SES are equal to 0.

**Final Results - Iteration 6 **

**Iterations stopped due to small change in likelihood function**

σ2
= 37.01402
Standard error of σ2
= 0.62449
τ
INTRCPT1,*β0* 2.64700
Standard error of τ
INTRCPT1,*β0* 0.39306

Random level-1 coefficient Reliability estimate
INTRCPT1,*β0* 0.751

The value of the log-likelihood function at iteration 6 = -2.328098E+004

**Final estimation of fixed effects: **

Fixed Effect Coefficient Standard

error *t*-ratio
Approx.
*d.f.* *p*-value
For INTRCPT1, *β0*
INTRCPT2, *γ00* 12.661175 0.148413 85.310 158 <0.001
MEANSES, *γ01* 3.674465 0.375431 9.787 158 <0.001

For SES slope, *β1*

INTRCPT2, *γ10* 2.191165 0.108660 20.165 7024 <0.001

Let’s take a look at the model.

MATHACH = γ00 + γ01MEANSES + γ10SES + u0+ r

The effect of SES on math achievement is defined by two sources. A source at the individual
level meant to *represent* his or her own individual circumstances (**SES**: family income,

occupation, etc.), and a source at the school level meant to *represent* the resources of a particular
school (**MEANSES**: mean of SES variable in each particular school). This SES effect is

captured jointly by two coefficients in the model: γ01 andγ10.

Despite that there are two coefficients in the model that capture the effect of SES, there are actually three effects to consider. There is the effect of the moving from a school with one SES to a school with a different SES, which is βb. But this effect is actually composed of the effect of

individuals’ SES on math achievement (βw) and the effect of schools, on the contexts’, effects on

math achievement (βc).

The coefficient for the variable at the individual level (γ10), **SES**, is always going to be the βw

effect. When we * grand* mean center

**SES**, what we capture with γ01 is the effect of schools’ mean

SES on math achievement, controlling for the effect of individual SES. This coefficient γ01

quantifies the effect ofbeing in a particular context, namely a particularschool with a particular
mean SES. Thus, γ01* IS* the contextual effect when the individual level component of that effect

is grand-mean centered. Together, both of these coefficients tell us about the effect of SES on math achievement in schools (the cumulative βb effect.) Sometimes, in using a grand mean

centered approach, it can be a bit more intuitive to think about the effects of context controlling for the effects of individual.

Finally, let’s look exactly at what these coefficients tell us. If Kid A and Kid B are in the same school, and Kid A’s SES is greater than Kid B’s SES by one unit, then Kid A’s MATHACH is greater than Kid B’s MATHACH by 2.19 (γ10).

If Kid A and Kid B have the same individual SES, but Kid A is in a different school that is greater by one unit of MEANSES, then Kid A’s MATHACH is 3.67 (γ01) greater than Kid B’s

MATHACH.

**Final estimation of fixed effects **
**(with robust standard errors) **

Fixed Effect Coefficient Standard

error *t*-ratio
Approx.
*d.f.* *p*-value
For INTRCPT1, *β0*
INTRCPT2, *γ00* 12.661175 0.148376 85.332 158 <0.001
MEANSES, *γ01* 3.674465 0.353014 10.409 158 <0.001

For SES slope, *β1*

**Final estimation of variance components **

Random Effect Standard
Deviation
Variance
Component *d.f.* χ
2 _{p}_{-value }
INTRCPT1, *u0* 1.62696 2.64700 158 670.25160 <0.001
level-1, *r* 6.08391 37.01402

**Statistics for the current model **

Deviance = 46561.966966

Number of estimated parameters = 5

**Contextual Effects Model (Group Mean Centered) **

Module: HLM2 (7.00) Date: Jun 15, 2011 Time: 05:51:39

**Specifications for this HLM2 run **

Problem Title: Contextual Effects Model (Group Mean)

The data source for this run = HSB.mdm

The command file for this run = C:\Documents and

Settings\manningma\Dropbox\Teaching\HLM Workshop 2011\HSB\Contextual Effects Model (Group Mean).hlm

Output file name = Contextual Effects Model (Group Mean).html The maximum number of level-1 units = 7185

The maximum number of level-2 units = 160 The maximum number of iterations = 1000

Method of estimation: full maximum likelihood

The outcome variable is MATHACH

**Summary of the model specified **

**Level-1 Model**

**Level-2 Model **

*β0j* = *γ00* + *γ01**(*MEANSESj*) + *u0j*

*β1j* = *γ10*

SES has been centered around the group mean.

SES has been centered around the mean for each school. So SES is equal to 0 at the mean for
each school. It might not be the same *value* for each school, but in the model it has the same

*meaning* for each school.

MEANSES has been centered around the grand mean.

**Mixed Model **

*MATHACHij* = *γ00* + *γ01***MEANSESj*

+ *γ10***SESij* + *u0j*+ *rij*

**Final Results - Iteration 6 **

**Iterations stopped due to small change in likelihood function**

σ2
= 37.01402
Standard error of σ2
= 0.62449
τ
INTRCPT1,*β0* 2.64694
Standard error of τ
INTRCPT1,*β0* 0.39306

Random level-1 coefficient Reliability estimate
INTRCPT1,*β0* 0.751

The value of the log-likelihood function at iteration 6 = -2.328098E+004

**Final estimation of fixed effects: **

Fixed Effect Coefficient Standard

error *t*-ratio
Approx.
*d.f.* *p*-value
For INTRCPT1, *β0*
INTRCPT2, *γ00* 12.647306 0.148410 85.218 158 <0.001
MEANSES, *γ01* 5.865602 0.359360 16.322 158 <0.001

For SES slope, *β1*

The contextual effect with group-mean centering of level-1 variable:

The model is exactly the same:

MATHACH = γ00 + γ01MEANSES + γ10SES + u0+ r

The only thing that we have changed is how we choose to center the within-unit variable, **SES**.
We have * group* mean centered it.

When we group mean center, coefficient γ01 is now the cumulative βb effect. So to get the effect

of context, we subtract the individual level effect, γ10, fromγ01. This underscores the fact that you

have to be mindful of your choice of centering when it comes to introducing effects that exist at both the aggregate and the individual levels, and further underscores the care needed in

interpreting your coefficients in these kinds of models.

**Final estimation of fixed effects **
**(with robust standard errors) **

Fixed Effect Coefficient Standard

error *t*-ratio
Approx.
*d.f.* *p*-value
For INTRCPT1, *β0*
INTRCPT2, *γ00* 12.647306 0.148448 85.197 158 <0.001
MEANSES, *γ01* 5.865602 0.320148 18.322 158 <0.001

For SES slope, *β1*

INTRCPT2, *γ10* 2.191172 0.129367 16.938 7024 <0.001

**Final estimation of variance components **

Random Effect Standard
Deviation
Variance
Component *d.f.* χ
2 _{p}_{-value }
INTRCPT1, *u0* 1.62694 2.64694 158 670.24237 <0.001
level-1, *r* 6.08391 37.01402

**Statistics for the current model **

Deviance = 46561.964487