Parameter Expanded, Non-centered Parameterization (PX-NC)

Parameter expansion (PX) is an additional strategy aimed at improving convergence of posterior sampling algorithms by reducing dependence between highly correlated parameters (Meng and Van Dyk, 1997; Meng and Dyk, 1999; Liu and Wu, 1999). The method builds upon data augmentation through the introduction of redundant (or working) parameters.

These parameters are only partially identifiable, as they are unidentified given only the observed data, but fully identified given the observed and augmented data (Gelman et al., 2008). Parameter expansion alters the joint distribution of observed data, y, and augmented data, θ, by introducing a one-to-one transformation on θ dependent on introduced working parameters α (e.g. φ = f (θ, α)) such that the marginal distribution of the observed data does not depend on the working parameters. That is, p(y|ψ, α) = p(y|ψ), where ψ

are unknown model parameters. We refer the reader to Gelman et al. (2008) for a very nice outline of parameter expansion for hierarchical linear and generalized linear models, as well as discussion of convergence rates of algorithms for these particular models. We give here a brief example of parameter expansion on the hierarchical random effects model following the outline of Gelman et al. (2008) to illustrate the strategy, and then present the implementation for Fourier-form DLMs.

Consider the following non-centered hierarchical group means model

Y_ij = µ + θ_j+ _i, _i ∼ N (0, σ_e²) (3.28a)

θ_j ∼ N (0, σ_θ²). (3.28b)

where Y_ij are the observed data and θ = (θ₁, . . . , θ_J)^T represent J group means and are considered the augmented data. Let ψ = (µ, σ²_θ, σ_e²). Then a standard DA MCMC sampling scheme alters between sampling θ given ψ, and ψ given θ in a Gibbs sampler. However, as discussed earlier, these standard MCMC schemes can be very inefficient, particularly when σ_θ² is small. Parameter expansion introduces the transformation ˜θ = θ/α, where α is a working parameter. Then, the parameter expanded model is

Y_ij = µ + α˜θ_j+ _i, _i ∼ N (0, σ_e² (3.29a)

θ˜_j ∼ N (0, σ²_θ˜). (3.29b)

where σ²_θ = α²σ_θ²_˜. An independent, conditionally conjugate prior for α is s the normal distri-bution (treating α like a regression coefficient) and for σ²_˜

θ is the inverse-gamma distribution.

The corresponding prior on σ_θ = |α|σθ˜ is half-cauchy (noting that this is the distribution of the absolute value of a normal random variable divided by the square root of an inde-pendent gamma random variable), and thus an MCMC scheme for sampling from the PX model is equivalent to sampling from the original model with a half-cauchy prior on σ_θ. It is important to note, however, that when priors on the working parameter and transformed parameters are independent, the parameters are not separately identifiable (e.g. α and σ_θ²_˜

in 3.29), and thus the backtransformations to the original model should be performed and inferences should only be made about original parameters. If inference about the parame-ters in the parameter expanded model are preferred (i.e. if the reparameterization is done not just for computational improvements), then careful specification of the priors on the working and transformed parameters can introduce identifiability (Gelman et al., 2008).

The motivation for implementing parameter expanded approach for the Fourier-form DLM is to develop a single MCMC sampling scheme that is at least as efficient as the standard and SC-NC sampling schemes for a larger region of the parameter space of (V , W ), as the standard and SC-NC schemes are efficient in separate areas of the parameter space (discussed further in Section 3.4.2). In the following, we present the PX-NC model for Fourier-form DLMs and show empirically through simulations that the parameter expanded approach performs at least as well as the standard MCMC sampler, and better than the SC-NC sampler, when evolution variances are non-negligible. This suggests the PX-NC model can be more universally applied to time series with varying degrees of evolution in seasonality than the SC-NC and standard parameterizations.

To derive the PX-NC model, we perform parameter expansion on the non-centered states θ˜_t = θ_t− G^tθ₀, introducing the working parameter vector . Under Model 1,  is a scalar, and the transformed states are θ^px_t = ˜θ_t/ and σ_η² = σ_w²/². Under Model 2, define L_ = blockdiag (₁I₂, . . . , _qI₂), and the transformation θ_t^px= L⁻¹_ θ˜_t. The transformed variances are σ²_ηj = σ_w²_j/²_j for j = 1, . . . , q such that W = Lblockdiag(σ_η²₁I2, . . . , σ²_η1I2)L. Then the

PX-NC Fourier-form models are as follows. For Model 1,

Y_t= X_tθ₀+ F θ_t^px+ v_t v_t ∼ N (0, σ²_e) (3.30a) θ_t^px= Gθ^px_t−1+ η_t η_t ∼ N (0, σ²_ηI) (3.30b)

θ₀ ∼ N (0, σ²_w0I) (3.30c)

σ_e ∼ Cauchy⁺(0, c) (3.30d)

 ∼ N (0, 1) (3.30e)

σ_η² ∼ IG(1/2, d²/2) (3.30f)

σ_w²

0 ∼ IG(a, b) (3.30g)

and for Model 2.

Y_t= X_tθ₀+ F L_θ_t^px+ v_t v_t ∼ N (0, σ²_e) (3.31a) θ_t^px= Gθ^px_t−1+ η_t η_t ∼ N (0, blockdiag(σ_η²

1I₂, . . . , σ_η²_qI₂)) (3.31b)

θ0 ∼ N (0, σ²_w0I) (3.31c)

σ_e ∼ Cauchy⁺(0, c) (3.31d)

_j ∼ N (0, 1) (3.31e)

σ_η²_j ∼ IG(1/2, d²/2) (3.31f)

σ_w²₀ ∼ IG(a, b) (3.31g)

The specific choices of priors on  and σ²_ηcorrespond to Cauchy⁺(0, d) priors on σ_wj = |_j|σ_ηj. We also specify the non-zero prior, θ^px₀ ∼ N (0, blockdiag(σ_η²₁I₂, . . . , σ_η²_qI₂) for Model 2, and θ₀^px∼ N (0, σ_η²I) under Model 1.

3.3.3.1 MCMC sampler for the PX-NC model

We implement a two-block DA Gibbs sampling scheme for the PX-NC Fourier-form models by choosing θ_0:T^px as the augmented data, and ψ₁ = (θ₀, , σ_e², σ²_η, σ_w²₀) and ψ₂ = (θ₀, , σ²_e, σ_η², σ_w²₀) as the unknown model parameters for Models 1 and 2, respectively. Then a DA MCMC scheme for Models 1 and 2 is implemented as follows (for ψl, l = 1, 2)

1. Draw θ_0:T^px from the conditional distribution π(θ_0:T^px|ψ_l, y_1:T) using FFBS.

2. Sample ψ_l jointly from the conditional distribution π(ψ_l|θ^px_0:T, y_1:T) in one block (con-ditioning on θ^px_0:T, y_1:T is implicit in the following):

(a) Sample (, θ₀), given σ_e, from the multivariate normal posterior N(m, C) with C = 1

(b) Sample σe, given ση,  and θ0, from its full conditional using Metropolis-Hastings.

(c) Sample σ_η²_j from an IG(c_η, C_ηj) where, for Model 1 that this transformation does not affect the sampling scheme, and can also be done as a post processing step.

Complete derivations of the PX-NC MCMC sampler are found in Appendix B.3. Under the PX-NC model, , σ_w and θ_1:T^px are not identifiable, and therefore the untransformed parameters should be used for inference.  and θ^px_1:T are unidentifiable up to a sign change (similarly to the SC-NC models), which only affects inference about σ_w. Again, this can be dealt with by implementing a random sign switch (with probability 0.5, multiply  and θ_1:T^px by -1) at each iteration of the sampler.