STK3100 and STK4100 Prediction of random effects Marginal models and generalized linear mixed models

STK3100 and STK4100

Covers the following material in chapter 9:

Department of Mathematics University of Oslo

Prediction of random effects Marginal models and

generalized linear mixed models

• Sections 9.1.2, 9.1.3, 9.3.2, 9.4.1, 9.4.2, 9.5.1, 9.5.2, 9.6.3, 9.6.4, and 9.7

Let be multivariate normally distributed

with mean vector and positive definite covariance matrix , i.e.

2

Conditional expectation for multivariate normal distribution (recap from chapter 3)

T 1 2

( , ,...., )Y Y Y_n

= Y

μ V Y ~ ( , )N μ V

Suppose is partitioned as Y

1 2

æ ö

= ç ÷ è ø Y Y

Y with and ¹

2

æ ö

= ç ÷ è ø μ μ

μ æ ¹¹₂₁ ¹²₂₂ ö

= çè ÷ø V V V V V Then

In particular

1 1

2 | 1 = 1 ~ N éë 2 + 21 11^- ( 1 - 1), 22 - 21 11^- 12 ùû Y Y y μ V V y μ V V V V

1

2 1 1 2 21 11 1 1

( | = ) = + ^- ( - )

E Y Y y μ V V y μ

where

and

3

Prediction of random effects

We will first see how we may predict the values of the random effects assuming that and are known We have

It follows that

u V = ZΣ Z_u ^T + R_ε

~ ( ,N _u)

u 0 Σ ε ~ ( ,N 0 R_ε) Note that

cov( , )Y u

= + +

Y Xβ Zu ε

T

~ é , æ +T öù

æ ö æ ö

ê ç ÷ú

ç ÷ ç ÷

è ø N ëè ø è ^{u ε u} øû

u u

ZΣ Z R ZΣ

Y Xβ

u 0 Σ Z Σ

β

cov( , )

= Zu ε u+ = Zvar( )u = ZΣ_u

4

By the result on conditional expectation for multivariate normal distributions, we have

T T 1

( | = ) = ( + ) (^- - )

E u Y y Σ Z ZΣ Z_{u u} R_ε y Xβ

We may now insert estimates for the unknown parameters and obtain the predictions

T T 1 ˆ

ˆ ˆ ˆ

ˆ = _u ( _u + _ε) (^- - ) u Σ Z ZΣ Z R y Xβ

The course web-page gives examples of R code (fecal fat and fev data)

Marginal models and GLMMs

We assume that are

independent vectors that corresponds to observations from each of n clusters

A marginal model with link function g has the form

for

[ ( )]_ij = ( )µ_ij = _ij

g E Y g x β

T

1 2

( , ,..., ) ; 1,...,

i = Y Yi i Yid i = n

Y

1,...., ; 1,..., i = n j = d

A generalized linear mixed model (GLMM) has the form

where the parameters are the fixed effects and give the random effects

[ ( | )]_{ij i} = _ij + _{ij i} g E Y u x β z u

~ ( , )

i N u

u 0 Σ

β

For a GLMM we have

( | )_{ij i} = ^-1( _ij + _{ij i}) E Y u g x β z u This gives

µ_ij = E Y( )_ij

where is the density function f u Σ( ;_{i u}) N 0 Σ( , _u) For the identity link we have

( ) ( ; )

µ^ij =

ò

( ; ) ( ; )

= ^{x βij}

ò

ò

= x βij

Thus we have an identity link (with the same effects) also for the marginal model, but a similar result does not

necessarily hold for other link functions [ ( | )]

= E E Y u_{ij i} =

ò

Consider first the mixed probit model for binary data

Models for a binary response

where and is the cdf of ( _ij =1| )_i = F( _ij + _i)

P Y u x β u

~ (0,s 2)

i u

u N

7

F z( ) Z ~ (0,1)N

Then

( _ij =1| )_i

P Y u

and 𝑃 𝑌_!" = 1 = ∫ 𝑃 𝑌_!" = 1 𝑢_! 𝑓 𝑢_!; 𝜎_#^$ 𝑑𝑢_! = + 𝑃(𝑍 − 𝑢_! ≤ 𝐱_!"β)𝑓 𝑢_!; 𝜎_#^$ 𝑑𝑢_!

( )

= P Z £ x β_ij +u_i = P Z u( - £_i x β_ij )

We have 𝑍 − 𝑢_!~𝑁 0,1 + 𝜎_#^$ , and hence (𝑍 − 𝑢_!)/ 1 + 𝜎_#^$~𝑁 0,1 , and so

𝑃 𝑌_!" = 1 = ϕ(𝐱_!"β/ 1 + 𝜎_#^$)

8

Thus the implied marginal model is also a probit model, but with replaced by x β_ij x β_ij / 1+s_u²

For the logistic mixed model

exp( )

( 1| )

1 exp( )

= = +

+ +

ij i

ij i

ij i

P Y u u

u x β

x β

the implied marginal model is not exactly of logistic form

But it is approximately a logistic model with replaced by where x β_ij / 1 (+ s_u / )c ²

x βij

»1.7 c

Example: Survey on attitude to abortion

Survey in the US on supporting legalized abortion under d = 3 three different situations

if person i supports legalized abortion under situation j , otherwise

ij 1 Y =

ij 0 Y = Summary of data:

When modelling the data, we need to take into account that the three responses for a person are dependent

A possible model is a logistic regression model with a random effect for persons:

10

logit[ (P Y_ij =1| )]u_i = x β_ij +u_i

Here , , and u_i ~ (0,N s_u²) β = ( , ,b b b b_{0 1 2}, )₃ ^T (1,1,0, ) for 1

(1,0,1, ) for 2 (1,0,0, ) for 3 ì =

= ïí =

ï =

î

i

ij i

i

s j

s j

s j

x

where if person i is a female and if person i is a male

i 1

s = s =_i 0

R code is given on course web-page

A mixed Poisson GLMM with log link is given by

Poisson models for correlated count data

where , and given we have that

are independent and Poisson distributed log éëE Y u( | )^{ij i} ù =û x β z u^ij + ^{ij i}

11

For the random intercept model [ ]

~ ( , )

i N u

u 0 Σ u_i

1, 2,...,

i i id

Y Y Y

log éëE Y u( | )^{ij i} ù =û x β^ij +uⁱ

~ (0,s 2)

i u

u N

we have (using the moment generating function) µ_ij = E Y( )_ij

exp( ) [exp( )]

= x β_ij E u_i

Thus the derived marginal model has the same effect of the covariates, but a different intercept

[ ( | )]

= E E Y u_{ij i} = E[exp(x β_ij +u_i)]

exp( s 2 / 2)

= x β_ij + _u

12

Maximum likelihood estimation for GLMMs

The likelihood function for a GLMM is given as

1 1

( , ; ) ( | ; ) ( ; )

= =

ì é ù ü

ï ï

= í ê ú ý

ï ë û ï

î þ

Õ Õ ò

!

n d

u ij i i u i

i j

f y f d

β Σ y u β u Σ u

This is a complicated expression, and the integral (typically) has no closed form solution

The integral has to be evaluated by numeric techniques Common approaches are Laplace approximations and Gauss-Hermite quadrature approximations

13

GEE estimation for marginal models

For cluster i with observations and means , the marginal model with link function g is

Let denote the working covariance matrix and let be the matrix with element

equal to

Remember the quasi-likelihood equations for univariate responses

∑_!"#^$ (𝜕𝜇_!/ 𝜕𝜂_!)^%𝑣(𝜇_!)^&#𝑥_!' 𝑦_! − 𝜇_! = ∑_!"#^$ (𝜕𝜇_!/ 𝜕𝛽_!)^%𝑣(𝜇_!)^&#(𝑦_! − 𝜇_!)

( )µ_ij = _ij

g x β

T

1 2

( , ,..., )

i = y yi i yid

y

Vi

T

1 2

(µ µ, ,...,µ )

i = i i id

μ

= ¶ / ¶

i i

D μ β d p´ ( , )j k

µ / b

¶ _ij ¶ _k

T ^-1( )

=

- =

å

Analogous for the multivariate setting we have the generalized estimating equations (GEE)

14

The GEE estimator is approximately multivariate normal with mean , and with covariance matrix

β ˆβ

1 1

T 1 T 1 1 T 1

1 1 1

var( )ˆ var( )

- -

- - - -

= = =

æ ö æ ö

= ç ÷ ç ÷

è

å

å

å

i i i

β D V D D V Y V D D V D

Analogous to the univariate case, we obtain sandwich

estimator for the covariance matrix by inserting estimates for the unknown parameters and by replacing by ^{var( )Y}_i

ˆ ˆ ˆ T

var( ) (Y_i = y_i -μ y_i)( _i -μ_i)

R code for attitude to abortion example is given on the course web-page