Generalized additive model GAM

Following the book ofWood(2017) the generalized additive model (GAM) was originally developed by Trevor Hastie and Robert Tibshirani in 1986. It is a generalized linear model with a linear predictor involving a sum of smooth functions of covariates. In similar way than in linear regression, this section introduces the GAM model for a single smooth function f (ti) and the GAM model considering a numeric tiand a binary zicovariates.

Furthermore, it is showed the model representations for compositional response models.

Generalized additive model for a single smooth function

In general the model for a single smooth function has a structure something like

g(µi) = Aiθ + f1(ti) + ei (3.19)

where µi ≡ E(yi) and yi ∼ EF (µi, φ), yi is a response variable, EF (µi, φ) denotes an exponential family

distribution with mean µi and scale parameter, φ, Ai is a row of the model matrix for any strictly parametric

model components, θ is the corresponding parameter vector, the f is the smooth function of the covariate tiand

eiare independent N (0, σ2) random variables. To solve this problem it is necessary both to represent the smooth

functions in some way and to choose how smooth it should be.

To estimate f requires that it be presented is such way that (3.19) becomes a linear model. This can be done by choosing a basis, defining the space of functions of which f (or a close approximation to it) is an element. Choosing a basis amounts to choosing some basis functions, which will be treated as completely known: if bj(x) is the jth

such basis function, then f is assumed to have a presentation

f (t) =

k

X

j=1

bj(t)βj (3.20)

for some values of the unknown parameters, βj. Substituting (3.20) into (3.19) clearly yields a linear model. It

should be noted that bj(t) is a piecewise linear basis of the univariate variable t and it is determined entirely by the

locations of the function’s derivative discontinuities, that is by the locations at which the linear pieces join up. Let these knots be denoted {x∗j : j = 1, ..., k} and suppose that x∗j > x∗j−1. Then for j = 2, ..., k − 1

bj(t) =            (t − t∗ j−1)/(t∗j− t∗j−1), t∗j−1≤ t ≤ t∗j (t∗_i+j− t)/(t∗ j+1− t∗j), t∗j ≤ t ≤ t∗j+1 0 otherwise (3.21)

So bj(t) is zero everywhere, except over the interval between the knots immediately to either side of t∗j · bj(t)

increases linearly from 0 at t∗_j−1to 1 at t∗_j, and then decreases linearly to 0 at t∗_j+1. Basis functions like this, that are non zero only over some finite intervals, are said to have compact support.

Finally, to represent the response compositional single covariate GAM model. The process is similar to linear regression. It only needs to be clear that it must be estimated D − 1 models for each coordinate. Even to make the model representation easier you can use β0instead of Aiθ

g(µ∗_ik) = β_0k∗ + f_1k∗(ti) + e∗ik (3.22)

where µ∗_ik ≡ E(x∗

ik) and x∗ik ∼ EF (µik,φ), x∗ikis the balance response variable (ilr), EF (muik, φ) denotes an

exponential family distribution with mean µikand scale parameter, φkfor each k = 1, 2, .., D − 1. f1kare different

smooth functions of the covariate tifor each k and eikare independent N (0, σ2) random variables.

Generalized additive model with interactions between a numeric covariate and a binary covariate The GAM model when it is included a numeric covariate tiand a binary covariate ziis

g(µi) = β0+ f1(ti) · zi+ β1· zi+ ei, yi∼ EF (µi, φi) (3.23)

Where zi is included in the model in two ways: a) it separates tiin two smooth functions (one for zi = 0 and the

other for zi= 1) and b) as a fixed effect of parameter β1, and g(µi), β0and yipreserve the properties of equation 3.19.

In the case of compositional response it is only changed yifor x∗ikwhere the parameter k = 1, 2, ..D − 1 represents

each coordinate of SD−1

g(µ∗_ik) = β∗_0k+ f_1k∗ (ti) · zi+ β1k∗ · zi+ e∗ik, x ∗

ik∼ EFk(µi, φi) (3.24)

Smooth predictors and estimation methods

In principle, using the R package mgcv it is possible to apply smooths of any number of predictors via four types of smooth:

• s() is used for univariate smooths, isotropic smooths of several variables and random effects.

• te() is used to specify tensor product smooths constructed from any singly penalized marginal smooths usable with s().

• ti() is used to specify tensor product interactions with the marginal smooths (and their lower order inter- actions) excluded, facilitating smooth ANOVA models.

• t2() is used to specify the alternative tensor product smooth construction which is especially useful for generalized additive mixed modelling with the gamm4 R package

All this smooth functions need a smoothing basis as it describes equation3.20. In practice there are a lot of basis and generally they are represented by a two letter character string. Table3.4.1shows the main smoothing bases considered when estimating the statistical models together with their advantages and disadvantages.

Table 3.4.1: Smoothing bases built into package mgcv, and a summary of their advantages and disadvantages (Wood,2006).

bs Description Advantages Disadvantages ’tp’ Thin plate regression

splines (TPRS)

Can smooth w.r.t. any number of covariates. Invariant to rotation of covariate axes. Can select penalty order

No ’knots’ and some optimality properties.

Computationally costly for large data sets.

Not invariant to covariate rescaling. ’ts’ TPRS with shrinkage As TPRS, but smoothness selection can

zero term completely

as TPRS

’cr’ cubic regression spline (CRS)

Computationally cheap.

Directly interpretable parameters

Can only smooth w.r.t 1 covariate. Knot based.

Doesn’t have TPRS optimality. ’cs’ CRS with shrinkage As CRS, but smoothness selection can zero

term completely.

As CRS ’cc’ cyclic CRS As CRS, but start point same as end point As CRS ’ps’ P-splines

Any combination of basis and penalty order possible.

Perform well in tensor products

Based on equally spaced knots. Penalties awkward to interpret. No optimality properties available.

About GAM fitting methods, the package mgcv uses six different ways

• GCV.Cp is the default method from mgcv R package. It uses a generalized cross validation for unknown scale parameter and Mallows’ Cp/UBRE/AIC for known scale.

• GACV.Cp is equivalent, but using generalized approximate cross validation GACV in place of GCV. • REML for REML estimation, including of unknown scale. This is a method of estimation in which estimators

of parameters are derived by maximizing the residual or restricted likelihood rather than the likelihood itself

Everitt and Skrondal(2010).

• P-REML for REML estimation, but using a Pearson estimate of the scale.

• ML An estimation procedure involving maximization of the like lihood or the log likelihood with respect to the parametersEveritt and Skrondal(2010).

All this methods have different properties and have some advantages and disadvantages between them. For instance GCV has the nice property that is invariant but it is not sensitive enough to over-fit. REML is similar to use Bayesian marginal likelihood and random coefficients have Gaussian distributions which most are familiar but asymptotically REML undersmooths relative to GCV resulting in higher asymptotic mean square error.