M ARKOV CHAIN M ONTE C ARLO (MCMC) SAMPLING

A simple example of an MCMC algorithm is the Metropolis algorithm, first introduced in 1953 [79]. From some starting value, the Metropolis algorithm requires a proposal distribution, 𝑃(𝜁), that is proportional to the target distribution, 𝑇(𝜁), and that can be evaluated for some parameter or set of parameters 𝜁. In Bayesian analysis, 𝑃(𝜁) is the product of the likelihood and the prior distribution (i.e. the numerator of Equation 2.7) and 𝑇(𝜁) is the posterior distribution. A symmetric distribution, 𝑔(𝜁|𝜁0), is used to generate a proposed move from the starting value 𝜁0.

In the simplest case, 𝑔(𝜁|𝜁0) may be a normal distribution where the mean is the initial starting

position 𝜁₀ and a standard deviation is specified. The ratio of 𝑃(𝜁) 𝑃(𝜁⁄ ₀) is compared to a draw from the uniform distribution between zero and one, 𝜐; if 𝑃(𝜁) 𝑃(𝜁⁄ ₀) is greater than or equal to 𝜐, then the proposed move is accepted and the value(s) of 𝜁 is saved as a sample in the MCMC chain. If 𝑃(𝜁) 𝑃(𝜁⁄ ₀) is less than 𝜐, the position is rejected and a new proposed move is generated

43 from 𝑔(𝜁|𝜁₀). A complete iteration or cycle of the algorithm concludes when a proposed move is accepted. An iteration of the Metropolis algorithm is summarized in Box 2.2.

Proposed moves in the Metropolis algorithm may be in one dimension or multiple dimensions. The Metropolis algorithm cycles through the above process until the probability of being at any given position 𝜁 resembles the probability of that position in the target distribution, 𝑇(𝜁). This final equilibrium is termed the stationary distribution. The cycles before reaching the equilibrium are termed the “burn-in”. Once the algorithm has reached the stationary distribution, additional cycles of the algorithm are used to collect samples for inference.

If no stationary distribution exists, the algorithm will not reach an equilibrium. A stationary distribution will not exist if the model is not identifiable; this often occurs in discrete mixture models where “label-switching” occurs between iterations of the algorithm [28].

Hastings expanded on the Metropolis algorithm in 1970 to the more general case where the condition of symmetry is not required for 𝑔(𝜁|𝜁0) resulting in the Metropolis-Hastings algorithm

[80,85]. The Metropolis-Hastings algorithm remains a powerful algorithm today as it can be employed to find the posterior distribution of virtually any defined model. This sampler is frequently used with logistic regression models.

The average ratio of accepted moves to proposed moves constitutes the acceptance rate and is largely responsible for determining whether an algorithm is efficient. When the acceptance rate is very low, as is often the case for more challenging models, this algorithm converges to the stationary distribution very slowly. A second important aspect of efficiency is whether the MCMC chain has good “mixing” through the various states of the model or parameter values; that is, that parameter values change states over the full range of the posterior distribution. Poor mixing can occur when there is multimodality in the posterior distribution [82].

When a closed-form solution is available for the conditional distribution of a single parameter 𝜁𝑗

conditional on all other parameters 𝜁_−𝑗 and the data 𝐷, 𝑓(𝜁𝑗|𝜁−𝑗, 𝐷), 𝑔(𝜁|𝜁0) is replaced with this

conditional distribution. The proposed move is necessarily in only one dimension while all other values 𝜁−𝑗 are held constant, but it is always accepted because the proposal distribution is

Box 2.2 Metropolis algorithm

1. Generate a proposed move from the initial values, 𝜁₀ to 𝜁 via a symmetric distribution 𝑔(𝜁|𝜁₀).

2. Calculate 𝛼 = 𝑃(𝜁) 𝑃(𝜁⁄ ₀).

3. Draw a value, 𝑈, from the uniform distribution between 0 and 1.

44 equivalent to the marginal posterior distribution of that parameter. This algorithm is termed a Gibbs sampler, and it is very efficient because the acceptance rate is one [86]. Linear regression typically uses a Gibbs sampler.

An advantage of MCMC algorithms is that they may be applied one parameter at a time or to a block of parameters so that the most suitable sampler is applied. This contributes to the modularity of Bayesian analysis. Modern software (Section 2.4.4) automates the selection of the best sampler for each parameter or block of parameters. Most models, all models implemented in this thesis, use multiple sampling algorithms together including those described above in addition to a variety of other fit-for-purpose samplers.

Sampling and convergence diagnostics

There are many methods which assist in determining whether an MCMC chain has converged to the stationary distribution. However, these tests are limited in that they can only detect lack of convergence rather than directly prove that convergence has occurred.

MCMC sampling methods can be sensitive to starting positions of the parameters. Additionally, for a single chain, a local optima may be encountered before the true stationary distribution is reached. Therefore, multiple chains are typically run from different starting positions (usually random) [28,82].

Gelman and Rubin’s convergence diagnostic, 𝑅̂, can be used to assess whether the chains have converged to the same distribution, presumably the stationary distribution [87]. 𝑅̂ is an estimate of the factor by which the scale of the current distribution of the parameters may be reduced if sampling were continued indefinitely. For a single parameter, the marginal posterior variance may be estimated by a combination of the within-chain variance 𝐴 and between-chain variance 𝐵 weighted by the number of posterior samples 𝑠, var(θ)̃ = (𝑠−1

𝑠 ) 𝐴 + 1 𝑠𝐵. 𝑅̂ is then estimated as 𝑅̂ = √((𝑠−1 𝑠 ) 𝐴 + 1

𝑠𝐵) 𝐴⁄ . A multivariate version was proposed in 1998 [88] and is applied in

this thesis.

The Geweke diagnostic [89], 𝐺̂, is used to assess within a single chain whether the chain has reached a stationary distribution for a single parameter. The Geweke diagnostic finds the difference in the posterior mean of the samples in the first 10% of the chain to the mean in the latter 50% of the chain and divides this difference in sample means by the estimated standard error to arrive at a Z-score.

Finally, the effective sample size (ESS) can be assessed to ensure sufficient samples are collected for inference [87]. This metric attempts to account for the correlation between MCMC samples within a chain for a given parameter. Where c is the number of chains and s is the number of

45 samples, the total number of samples 𝑐𝑠 are not independent (this is called “autocorrelation”) and must be weighted by an estimate of the correlations between samples:

2.10 ESS = 𝑐𝑠 1+2∑𝑇 (1−(_2var(𝜃)𝑉𝑡_̃ )) 𝑡=1 , 𝑉_𝑡= 1 𝑐(𝑠−𝑡)∑ ∑ (𝜃𝑎,𝑏− 𝜃𝑎−𝑡,𝑏) 2 𝑠 𝑎=𝑡+1 𝑐 𝑏=1

where 𝑉_𝑡 is the variogram at each updating of the algorithm 𝑡.

Scaling of regression covariates, and the outcome if continuous, enables better estimation of the priors for the regression coefficients and improves the speed of convergence of the MCMC chains. Where there is high collinearity between covariates in regression modelling, MCMC will converge slowly; this may be addressed by centering the covariates at zero [28].

Estimation of variance

In this thesis, Bayesian inference is performed using posterior means and 95% credible intervals (CrIs) instead of traditional CIs to quantify uncertainty. The 95% CrI expresses the range over which 95% of the probability distribution lies, or having a 95% probability the true value lies therein. In contrast, the frequentist CI expresses the range wherein, were the test repeated many times, it is anticipated that the true value would lie between the values in 95% of results. An equi- tailed 95% CrI can be obtained from the 2.5 and 97.5 percentiles of the posterior distribution. Alternatively, a highest posterior density (HPD) 95% CrI can be obtained from the shortest interval for which the posterior probability for that region is 0.95 [90]. In this thesis, HPD 95% CrIs are obtains from MCMC sampling estimates.

CrIs are more intuitive in their meaning than CIs and are better able to describe non-symmetric distributions. However, frequentist properties of the Bayesian estimates can be determined as well [91].

Extensions to non-linear functional forms of the predictor

Bayesian analysis using MCMC can be extended easily to models wherein the functional form of the error-prone predictor is non-linear. Examples where the predictor takes a polynomial form or even a smoothing function have been published elsewhere [1,53,92]. Bayesian modelling using MCMC when the substantive model includes a squared transformation of the error-prone exposure is included in Chapter 4. Also in Chapters 4-6, a hybrid method combining Bayesian modelling and properties of RC will be presented for use with polynomial transformations of the error-prone exposure in the substantive model. In Chapter 5, I will also introduce a novel application of Bayesian variable selection methods to the selection of the best polynomial transformation of the error-prone exposure.

46 Software

Bayesian analysis with MCMC sampling can be performed without advanced knowledge of the algorithms or derivation of the full likelihood distribution using software packages such as JAGS [62], OpenBUGS [93], and WinBUGS [94]. Each can be implemented in stand-alone software; alternatively, one can use R packages such as “rjags”, “BRugs” or “R2WinBUGS” to call each software, respectively [95,96]. In this thesis, I use “rjags” to call JAGS for all implementations using MCMC and the “coda” package to perform convergence diagnostics [97]. Once the model and prior distributions are specified in the JAGS language, the JAGS software automatically selects the appropriate samplers for each parameter or block of parameters.