rameters P increases. For estimating the parameters of the proposed SEV VonB G-Normal mixture model this work increases as the number of groups, G, increases. Alternatively, the Template Model Builder (TMB) frees the statistician from the task of writing, testing and maintaining derivative code.
In the next section, we discuss the TMB package and its implementation of our proposed approach to estimate parameters.
3.4
Template Model Builder
TMB (e.g. Skaugh and Fournier, 2006 [41]; Wang, 2015 [52]) is a free and open source R package (R Core Team, 2014 [34]) that is designed for estimating complex nonlinear random effects models. One needs to define the joint log-likelihood function of the data and the random effects as a C++ template function. Other operations such as integration and calculation of the marginal score function are done in R. This package evaluates and maximizes the Laplace approximation of the marginal likelihood where the random effects are automatically integrated out. This approximation, and its derivatives, are obtained using automatic differentiation of the joint likelihood.
3.4.1
Automatic Differentiation
Automatic Differentiation (AD) (e.g. Fournier et al., 2012 [18]), also known as com- putational differentiation or algorithmic differentiation, is a set of techniques that nu- merically differentiate a function, which frees us from calculating and incorporating the derivatives. Two methods, “source transformation” and “operator overloading” are commonly used to implement automatic differentiation. C++ automatic differen- tiation (CppAD) (e.g. Bell, 2012 [5]) implements the operator overloading approach.
3.4. TEMPLATE MODEL BUILDER 57 This approach is easier to implement and use than “source transformation”. The TMB R package uses CppAD to provide up to third order derivatives of the joint likelihood function that the user writes in the C++ template. These derivatives are required for the Laplace approximation of the marginal likelihood.
3.4.2
Laplace Approximation
The Laplace approximation (e.g. Skaugh and Fournier, 2006 [41]) is used to ap- proximate the intractable integral in the marginal likelihood (Eqn. 3.10). Let y = (y1, y2, ..., yn) be the vector of response variables, Λ = (Λ1,Λ2, ...,Λn) be the vector
of unobserved random effects, and let θ = (θ1, θ2, ..., θm) be the vector of fixed effects
parameters. Moreover, fθ(y | Λ) denotes the conditional pdf of y given Λ, and fθ(Λ)
denotes the marginal pdf of random effects Λ. The marginal likelihood function for θ is defined by integrating out the random effects Λ from the joint pdf of y and Λ, fθ(y | Λ)fθ(Λ), such that L(θ) = Z fθ(y | Λ)fθ(Λ)dΛ = Z eh(Λ,θ)dΛ, (3.10) where h(Λ, θ) = log{fθ(y | Λ)} + log{fθ(Λ)} is the joint log-likelihood. The main
computational challenge is in computing the integral in Eqn. (3.10), when there is no analytical solution. TMB uses the Laplace approximation for Eqn. (3.10), which yields the marginal likelihood approximation
L∗(θ) = det{H(θ)}−1/2eh( ˆΛ(θ),θ), (3.11) where ˆΛ(θ) = arg max
Λ {h(ˆΛ(θ), θ)}, H(θ) = ∂2
∂Λ2h(Λ, θ)|Λ= ˆΛ(θ),and det{H(θ)} denotes the determinant of H(θ). The term eh( ˆΛ(θ),θ) in Eqn. (3.11) is a profile likelihood,
3.4. TEMPLATE MODEL BUILDER 58 which treats the random effects as nuisance parameters and θ as the parameters of interest. The Hessian, H, is evaluated by CppAD. Using the AD and Laplace approximation simplifies the parameter estimation of hierarchical models. The TMB user needs only to specify the joint log-likelihood function h(Λ, θ). TMB uses the Cholesky decomposition of H(θ); therefore, the Laplace approximation is well-defined only if h(Λ, θ) is positive definite.
In an R session, we read in the data, dynamically link the C++ function template, set up the initial values for θ, specify the random effects, and optimize the objective function. TMB automatically provides a standard error report for ˆθ, and also any differentiable function of θ, φ(θ) that the user specifies, by using the δ-method:
Var(φ(ˆθ)) = −nφ′(θ)[∂ 2{logL∗(θ)} ∂θ∂θ′ ] −1φ(θ)o θ=ˆθ .
3.4.3
SEV VonB G-Normal Mixture Model Implementation
In this section, we describe the implementation in TMB of the SEV VonB G-Normal mixture model. The observed likelihood (Eqn. 3.6) is re-written as
LA(θ | Y, X ) = Z
eh(XT,θ)d XT,
where h(XT, θ) = log{fY|XT(Y | XT; θR)} + log{fX |XT(X | XT; σu)} + log{f
A
XT(XT; θE)}, which is the joint log-likelihood. For convenience we estimate the logarithm of L∞, k,
and σ, which can take values between (−∞, ∞) whereas L∞, k, and σ vary between
(0, ∞). Let
θ = (log(L∞), log(k), a0, λ, µd,log(σ)).
The vector of random effects are XT. We first specify the negative joint likelihood
3.4. TEMPLATE MODEL BUILDER 59 for an SEV VonB G-Normal mixture model then calculates the marginal likelihood function for θ using the Laplace approximation. The final step is to optimize this objective function in R.
C++ Template Code: The user template for the negative joint log-likelihood (the file named fitmix.cpp) is given in Appendix B (section B.2). The first four lines are standard and should be the same for most models. The first line includes the TMB specific macros and functions, including dependencies such as CppAD and Eigen. The following three lines are the syntax for starting a function template, where Type is a template parameter that the compiler replaces by an AD type that is used for numerical computations. Lines 5-10 declare the vector of variables to be the same as tmb.data in the R session included at Appendix B (section B.3). For example, in line 5 DATA VECTOR(age) declares the vector age to be the same as tmb.data$age in the R session. Lines 11-17 include the parameters to be the same as parameters in the R session. For example, line 11 PARAMETER(log Linf) declares the scalar log Linf to be the same as parameters$log Linf in the R session. Line 14 PARAMETER VECTOR(lambda) declares the vector lambda to be the same as parameters$lambda in the R session. The other scalar parameters are declared in a similar manner. Note that the user template does not distinguish between the fixed parameters and random effects but rather codes them both as parameters.
Lines 33-37 include the code for estimating λ, which are the continuation ratio logits of probabilities for groups. Lines 38-42 include the pdf of a G-normal mixture distribution. Lines 43-47 include the negative joint log-likelihood. Lines 48-53 include the template syntax which reports the parameters back to R with derivatives. For example, line 48 ADREPORT(Linf) reports the Linf back to R with derivatives. Lines 54-61 include the template syntax which report the parameters back to R. The last
3.5. SIMULATION STUDIES 60