Part I. Bayesian regularization in the AFT model
6. MCMC inference for the extended AFT model
6.3. Update of the parameters
The Markov chain is generated via MCMC simulations based on drawing from the full conditionals of parameters or parameter blocks given the remaining parameters and the data as derived in the previous sections. The methods are implemented in the R-function baftpgm() which will be provided from the author on request. The usage of the function is described in the Appendix D.5.
6.3.1. Preprocessing
Standardization: To ensure that comparable regression coefficient sizes imply comparable effect sizes, the covariates are standardized in advance. This avoids the extensive covariate-specific tuning of the priors for different covariate scales. We standardize covariates with linear effects to zero empirical mean and unit empirical variance. To obtain that smooth covariates taking values in [ 1,1]− , we can apply the transformation
ij j,min * ij j,max j,min 2(z z ) z 1 z z − = − − .
Starting values: In general we avoid preprocessing steps to fit the model in order to obtain suitable starting values. An automatic computation of starting values is not implemented in the function baftpgm() and in our simulations and applications we start with weakly specified models. The accurately starting values and prior specifications are given corresponding sections. If desired, starting values can e. g. be computed by a view iterations with the R-function bayessurvreg2{bayesSurv}. 6.3.2. Pseudocode
[1] Initialization:
Specify the PGM: Set number g of Gaussian basis functions and choose the location of the 0 means m , the scales j s and the order 2j d of the random walk prior for the (transformed) 0 mixture weights. Select the hyperparameters h ,h1,σ 2,σ to specify the inverse gamma prior for the
scale parameter σ.
Specify the regularization priors of the linear effects: Set the values of the hyperparameters 1, ,
lasso prior. For the Bayesian NMIG prior set the values v , v0 1 of the indicator Ij, set the values of the hyperparameters h ,h1,ψ 2 ψ, of the inverse gamma prior for the variance parameter ψ2j and set the hyperparameters h ,h1,ω 2,ω of the beta prior for the complexity parameter ω.
Specify the non-linear effects: Set number gj of B-spline basis functions and choose the order j
d of the random walk prior for basis function weights. Select optionally variants described in Subsection 6.2.
Standardize the covariates according to Section 6.3.1 and choose appropriate starting values for the parameters ( , , , 2 , 2, , )
α′ β′
′ ′ ′ ′ ′
= σ
θ α β γ τ τ ρ .
Set the number C of iterations, set c 0= and repeat the following steps until c C< . [2] Update of the unregularized linear regression coefficients:
Draw a new value γ(c 1)+ from a multivariate Gaussian full conditional with mean vector and
covariance matrix given in (6.3).
[3] Update of the regularized linear regression coefficients:
Draw a new value β(c 1)+ from a multivariate Gaussian full conditional with mean vector and
covariance matrix given in (6.4).
[4] Update of the shrinkage- and selection-prior components:
Bayesian ridge (A): Draw a new value of the complexity parameter (c 1) j
+
λ from the conditional gamma distribution given by (6.7) and set the variance parameter 2,(c 1)j (c 1)
j 1 2
+ +
β
τ = λ , j 1,..., p= x. Bayesian ridge (B): Draw a new value of the complexity parameter λ(c 1)+ from the conditional
gamma distribution given by (6.8) and set the variance parameter 2,(c 1)+ 1 2 (c 1)+ β
τ = λ .
Bayesian lasso: Draw a new value of the variance parameter 2,(c 1)j + β
τ , j 1,..., p= x, from the conditional inverse Gaussian distribution given by (6.9). Draw a new value of the complexity parameter λ2,(c 1)+ from the conditional gamma distribution given by (6.10).
Bayesian NMIG: Draw a new value of the indicator I(c 1)j+
β , j 1,..., p= x, from the conditional Bernoulli distribution given in (6.11). Draw a new value of the variance parameter 2,(c 1)j +
β
ψ ,
x
j 1,..., p= , from the conditional inverse gamma distribution given in (6.12). Draw a new value of the complexity parameter (c 1)+
ω from the conditional beta distribution given in (6.13). [5] Update of the regularized spline coefficients of the nonlinear effects:
Draw a new value (c 1) j+
α , j 1,..., p= z, from a multivariate Gaussian full conditional with mean vector and covariance matrix given by (6.5).
To center the functions, compute the mean of the function evaluations at the observed data points (c 1) 1 n (c 1) j,k ij j i 1 j,k c + n− + B (z ) = =
∑
α .Adjust the current states of (c 1) j + α by (c 1) (c 1) j cj + + −
α , j 1,..., p= z, and adjust the intercept γ(c 1)+ by z (c 1) (c 1) (c 1) 1 p c + ... c + + γ + + + .
[6] Update of the smoothing variances associated to the spline coefficients: Draw a new value of the variance parameters 2,(c 1)j +
α
τ , j 1,..., p= z, from the conditional inverse gamma distribution given by (6.14).
[7] Update of the scale parameter:
Draw with slice sampling a new value of the variance parameters σ2,(c 1)+ from the conditional
inverse gamma distribution given by (6.28) with (6.27). [8] Update of the transformed mixture weights:
Option 1: (Method “mhcond”) Draw a new value αɶ(p)0 = α( (p)0,1,...,α0,k 1(p)−,α0,k 1(p)+,...,α0,g(p))′ from a multivariate Gaussian proposal distribution with mean vector and covariance matrix given by (6.18). Accept the proposed state as new state of the chain with the acceptance probability given in (6.19). If the proposal is accepted, set (c 1) (p)
0+ = 0
α α , else set (c 1) (c) 0+ = 0
α α .
Option 2: (Method “mhmarg”) Draw a new value αɶ(p)0 = α( (p)0,1,...,α0,k 1(p)−,α0,k 1(p)+,...,α0,g(p))′ from a multivariate Gaussian proposal distribution with mean vector and covariance matrix given by (6.20). Accept the proposed state as new state of the chain with the acceptance probability given in (6.19). If the proposal is accepted, set (c 1) (p)
0 0 + = α α , else set (c 1) (c) 0 0 + = α α .
Option 3: (Method “mcondblock”) Draw a new value αɶ(p)0 = α( (p)0,1,...,α(p)0,k 1−,α0,k 1(p)+,...,α0,g(p))′ from the Gaussian proposal distribution with mean vector and covariance matrix given by (6.21) and accept the proposed state as new state of the chain with the acceptance probability given in (6.22). If the proposal is accepted, set (c 1) (p)
0+ = 0
α α , else set (c 1) (c) 0+ = 0
α α .
Option 4: (Method “mcondstep”) Draw a new value αɶ(p)0,ℓ from the multivariate Gaussian proposal distribution with mean vector and covariance matrix given by (6.21) and the stepwise update. Accept in each step ℓ=1,...,g0−1, the proposed state as new state of the chain with the acceptance probability given in (6.23). If the proposal is accepted, set (c 1) (p)
0+ = 0, α α ℓ, else set (c 1) (c) 0 0, 1 + − = α α ℓ .
Option 5: (Method “slice”) Draw a new value with slice sampling to update each component (c 1)
0, j+
α , j 1,..., k 1, k 1,...,g= − + 0, from the conditional distribution given by (6.25).
Option 6: (Method “dirichlet”) Draw a new value state of the component weights w(c) from the Dirichlet distribution (6.26).
Skip the update of the smoothing variance parameter in [9].
Option 7: (scalebasis=TRUE) To standardize the error distribution to zero mean and unit variance, compute the mean and variance of the error distribution at the current values of the basis knots (c)
j
m and basis variances 2,(c) j s
(
)
0 0 g (c 1) (c) g (c 1) 2,(c) 2,(c) (c 1) 2,(c 1) 2,(c 1) j 0 j j 0 j j j 1w ( )m , j 1w ( ) m s + + + + + ε = ε = ε µ =∑
α σ =∑
α + − µ .Update the basis means and variances according to (6.31) with
(c) (c 1) 2,(c) j j (c 1) 2,(c 1) j (c 1) j 2,(c 1) m s m , s + ε + + + + ε ε − µ = = σ σ , j 1,...,g= 0.
Finally, adjust the intercept (c 1) (c 1) 2,(c 1) (c 1)
0+ 0+ + ε+
γ → γ + σ µ and the scale parameter
2,(c 1)+ 2,(c 1)+ 2,(c 1)+
ε ε
σ → σ σ according to (6.32).
[9] Update of the smoothing variance of the mixture weights: Draw a new value of the variance parameter 2,(c 1)0 +
α
τ from the conditional distribution given by (6.14).
[10] Update of the latent component labels:
Draw a new value of the latent component labels (c 1)
0 i
r + ∈{1,...,g }, i 1,..., n= , from the conditional multinomial distribution with probabilities given by (6.29). Update the number of observations in the mixture components (c 1)
{
(c 1)}
j i
n + # i : r + j
= = , j 1,...,g= 0. [11] Update of the latent exact log-survival times:
For the censored observations i {1,..., n}∈ draw a new value of the latent log-survival time (c 1) i y +
from the truncated Gaussian distribution given by (6.30). 6.3.3. Postprocessing
Standardization of the error: As outlined in Subsection 6.2.1 the unconstrained estimation of the baseline error is leading to the non-identifiability of the parameters defining the baseline error distribution Y0= γ + σε0 and the involved parameters do not show the desired convergence. When the basis function means m and standard deviations k s are fixed during the sampler k (scalebasis=FALSE), we have to compute the standardized error density in post-processing steps utilizing the obtained MCMC sample. We do this by applying the same transformation of the mean
k
m and standard deviation s as described in Option 7 to the sampled values of the location and scale k parameter. The resulting, adjusted sample values of the intercept γ0 and the scale parameter σ show the desired convergence.
Recomputation of the weights: With the procedure described in Option 7 we achieve a standardized error distribution, ε0~ PGM(0,1), but the resulting variability in the locations mɶk and scales sɶk of the Gaussian basis function sometimes prevents the direct detection of the convergence of basis function weights. Finally, to show in addition the convergence of the weights, we compute the estimated standardized version of error density f ( )ε0 ⋅ at a fixed number of grid points. For the standardized densities we can e. g. use the starting knots m=(m ,..., m )1 g0 ′ of the Gaussian basis densities, but any set of grid points is possible. With respect to (6.33), we have to solve the constrained linear equation system (s) (s) (s)0
ε = B w f subject to (s) k wɶ >0 and g0 (s) k k 1= w =1
∑
ɶ , where 0 (s) (s)2 (s) 1 i, j g k k ( (m | m ,s ))≤ ≤ = ϕB ℓ ɶ ɶ denotes the matrix of the Gaussian basis functions
(s) (s)2 k k ( | m ,s ) ϕ ⋅ ɶ ɶ with adjusted mean (s) k
mɶ and standard deviation (s) k
sɶ , k 1,....,g= 0, evaluated at each grid point mℓ,
0 1,...,g = ℓ . Further (s) (s) (s)0 1 g (w ,..., w )′ =
wɶ ɶ ɶ is the vector of recomputed basis function weights and 0 0 0 0 (s) (s) (s) 1 g (f (e ),...,f (e )) ε = ε ε ′
f is the vector of the standardized error density computed with the parameter values of the s-th iteration. Since some of the border weights are often very close to zero, solving the constrained equation system becomes often numerically instable. To approximate the solution, we replace one component of the system (s) (s) (s)
0
=
B w f to satisfy g0 (s) k k 1= w =1