Generalized linear mixed models and its estimation

Clayton (1996) gives a very good overview of generalized linear mixed models(GLMMs). The special case of GLMMs without random effect all known as generalized linear models (GLMs). The random effects not only determine the correlation structure between observations on the same group, they also take account of heterogeneity a- mong groups that is attributed to unobserved factors. We will start with generalized linear models and then extend GLMs to GLMMs by adding random effects. Different inference methods for GLMMs will be introduced.

Generalized linear mixed models

Generalized linear models (GLMs) extend the linear model to allow distributions from the exponential family. The outcome of the response variables, Y = (y1, . . . , yn)0, is

assumed to be generated from a particular distribution function in the exponential family. The mean, µ≡(µ1, . . . , µn)0, of the distribution depends on the explanatory

variables, X ∈ Rn×p_{, through:}

E(Y) = µ=g−1(Xβ)

whereE(Y) is the expected value ofY,β= (β1,· · · , βp);Xβ is the linear predictor,

a linear combination of unknown parameters,β; g is the link function. The unknown but fixed regression parameters inβare estimated by solving the maximum likelihood equations. Generalized linear models (GLMs) consists of three components: A distri- bution functionf from the exponential family; a linear predictorη =Xβ; and a link functiong such thatE(Y) = µ=g−1(η). The link function provides the relationship between the linear predictor and the mean of the distribution function. There are many commonly used link functions, such as probit and logit link function in the Bernoulli case and log-link function in the Poisson case. Here we give the common response functions (g−1) for Bernoulli-type GLMMs.

• Probit Φ(x) = (1/p2π) Z x −∞ exp(−u2/2)du • logit ex/(1 +ex) = 1/(1 +e−x) • complementary log-log 1−exp(−ex)

Generalized linear mixed models (GLMMs) are GLMs with one or more random effects. The linear predictor should be rewritten as

η=Xβ+Zψ

where the fixed effectβ remains the same as in GLMs and random effects ψ are ran- dom variables drawn from a distribution. The random effects generate heterogeneity

beyond that which can be captured with fixed effects. We firstly consider a single shared random effect in every time period in our model. Then we consider different random effects for different sectors. This will capture additional variability associated with economic effects in different sectors. X are fixed vector and could be rating and macroeconomic covariates in credit risk models,β are the corresponding fixed effects. While Z are random vectors and could be year and industry sectors, ψ are random effects.

Estimation of GLMMs

We can estimate the GLMMs parameters using Markov Chain Monte Carlo(MCMC) methods from a Bayesian point-of-view. Bayesian MCMC approach allows us to handle more complex models than standard packages. Furthermore, the Bayesian approach can work well for sparse default data. For a Bayesian approach to GLMMs of portfolio credit risk, see McNeil and Wendin (2007) for more information. However, some standard package like S-plus and R provide some defined functions for estimation of GLMMs. The advantage of these defined function is that they are simple and very easy to handle. These defined functions use maximum likelihood inference. We will give more details about maximum likelihood inference in the next paragraph.

Maximum likelihood (ML) inference for GLMMs is only really a possible option for the simplest model. The unconditional distribution is obtained by integrating the random effects. Factor models have conditional independence property, if we write

pYt,i|Ψt(y|ψ) for the conditional probability mass function of Yt,i given Ψt,

L(β, σ;data) = Z · · · Z n Y t=1 mt Y i=1 pYt,i|Ψt(Yt,i |ψ) f(ψ1, . . . , ψn)dψ1. . . dψn

wheref denotes the joint density of the random effects. With theiidGaussian random effects with marginal Gaussian densityfΨ, we can reduce then−dimensional integral

to L(β, σ;data) = n Y t=1 Z mt Y i=1 pYt,i|Ψt(Yt,i |ψ)fΨ(ψt)dψt

with the product of one-dimensional integrals. This can be easily solved the unknown parameters.

For those complicated models where the exact likelihood function is difficult to com- pute, approximation becomes unavoidable. There are several methods including pe- nalized quasi-likelihood (PQL), marginal quasi-likelihood(MQL), Laplace and adap- tive gaussian quadrature. More details about PQL and MQL can be found in Breslow and Clayton (1993).

We use the glme function in the S-Plus correlated data library for the following analysis. The R functionglmmPQLis an alternative choice. However, the R function

glmmPQL can be treated as a special case of glme with (RE)P QL method. In S- plus glme is a more general function; there are four different methods that can be used in fitting models and these are “AGQUAS”,“LAPLACE”, (restricted) penalized quasi-likelihood ((RE)PQL) and (restricted) marginal quasi-likelihood ((RE)MQL) method. However, (RE)PQL and (RE)MQL gives similar results. Methods AGQUAD and LAPLACE are restricted to family binomial(“logit”) or poisson(“log”). The “probit” link function makes it straightforward to calculate default probability for our one factor model in this analysis, we choose to use the default methods which are “(RE)PQL”.