Assume that we have a random variable of interest X, taking values inX . Its probability distribution π(dx) has a density onX with respect to a dominating measure dx, which is abusively denoted as π(x)1. The value of X, denoted by x, is observed indirectly through
an observation process generating values Y ∈ Y according to conditional observation probability distribution who also has a density onY with respect to dy, which is denoted as g(y|x). The density g(y|x) is also called the likelihood. The posterior distribution of X given Y = y has the following density which is given by Bayes’ theorem
π(x|y) = R π(x)g(y|x)
X π(x′)g(y|x′)dx′
.
Approximate Bayesian computation (ABC) deals with the problem of Monte Carlo ap-
proximation to π(x|y) when the likelihood g(y|x) is intractable. By intractability it is meant either that the density does not have a close form expression or that it is pro- hibitive to calculate it. ABC methods try to approximate π(x|y) without circumventing the calculation of g(y|x) and for this reason they are also known as likelihood-free meth- ods. The main idea behind ABC is simulating from the observation process and accepting simulated samples provided that they are close to the observed value y in some sense. ABC methods have appeared in the past ten years as one of the most satisfactory ap- proach to intractable likelihood problems. This section is a brief and limited review of the main contributions to the ABC methodology, for a more detailed recent review, one
1
can see Marin et al. [2011].
The idea core to ABC is first mentioned in Rubin [1984]; but the first ABC method was proposed by Tavar´e et al. [1997] as a special case of rejection sampling for discrete Y. It proposes to sample (x, y) from π(x)g(u|x) and consider only those samples for which u = y. It is not difficult to show that if the accepted samples are (X(1), y), . . . , (X(N ), y),
then X(1), . . . , X(N ) are samples from the posterior π(x|y). Note that this a rejection
sampling method for the distribution π(x, u|y) on X × Y, which is given by
π(x, u|y) ∝ π(x)g(u|x)Iy(u) (2.20)
and when this density is integrated over u, we end up with π(x|y). This method is exact in the sense that the obtained samples for X are drawn from π(x|y). However, obviously Iy(u) would not work for continuous Y, since the probability of hitting {y} will be zero. The first genuine ABC method, proposed by Pritchard et al. [1999] as a rejection sampling method also, relaxes Iy(u) and replaces (2.20) with
πǫ(x, u|y) ∝ π(x)g(u|x)IAǫ
y(u) (2.21)
where Aǫ
y is called the ABC set and defined based on some summary statistic s : Y → Rds
and a distance metric ρ : Rds × Rds → R as
Aǫy ={u ∈ Y : ρ[s(u), s(y)] < ǫ}. (2.22) If s is sufficient with respect to x, one can show that as ǫ tends to zero, the marginal of density πǫ(x, u|y) with respect to x converges to the posterior π(x|y). In most cases
sufficient statistics are not available, hence the choice of summary statistics is of great importance. The ABC literature is rich in papers discussing on the selection of these sufficient statistics, see Fearnhead and Prangle [2012] for an example. The ABC method in Pritchard et al. [1999] is also a rejection sampling method, targeting πǫ(x, u|y): generate
(X, U) from π(x)g(u|x) and consider only those samples for which U ∈ Aǫ
y. This method
is summarised in Algorithm 2.8.
Algorithm 2.8. Rejection sampling for ABC: To generate a single sample from πǫ(x, u|y),
1. Generate (X, U)∼ π(x)g(u|x).
2. If U ∈ Aǫ
y, accept (X, U); else go to 1.
Using simulations from the prior distribution π(x) can be inefficient since this does not take neither the data nor the previously accepted samples into account when proposing a new x and thus fails to propose values located in high posterior probability regions.
To overcome this impracticality of rejection sampling, an MCMC based ABC method was developed by Marjoram et al. [2003]. The method is simply an MCMC algorithm targeting πǫ(x, u|y) which uses an instrumental kernel with density q(x′|x)g(u′|x′) to
move samples (x, u) and takes either (x′, u′) or (x, u) as the next sample according to the
corresponding acceptance probability
α(x, u; x′, u′) = min 1,q(x|x ′)π(x′)I Aǫ y(u ′) q(x′|x)π(x) , x, x′ ∈ X , u, u′ ∈ U.
The MCMC-ABC method is given in Algorithm 2.9.
Algorithm 2.9. MCMC for ABC: Begin with some (X1, U1) ∈ X × U. For n =
2, 3, . . .
• Generate (X′, U′)∼ q(x′|x)g(u′|x′).
• Set (Xn, Un) = (X′, U′) with probability α(Xn−1, Un−1; Xn, Un); otherwise set (Xn, Un) =
(Xn−1, Un−1).
It is useful to interpret the ABC posterior all in terms densities: We can consider IAǫ
y(u) as to which the density of the conditional distribution of Y given U = u, say
κǫ(y|u), is proportional to. Then we can rewrite (2.21) as
πǫ(x, u|y) =
π(x)g(u|x)κǫ(y|u)
R
X ×Yπ(x′)g(u′|x′)κǫ(y|u′)dx′du′
.
A useful generalisation of πǫ(x, u|y) can be made by taking κǫ(y|u) some normalised kernel
with bandwidth ǫ centred at u. In many applications, it is practical sometimes to take κǫ proportional not to an indicator function but to a smooth kernel, such as a Gaussian
kernel, to make calculations tractable or to avoid computational waste due to rejections. We will see a use of choosing a smooth kernel in Chapter 6.
Another use of kernels in defining the ABC posterior is to be able to express the difference between the ABC posterior and the real posterior in terms of model error. Note that the ABC suffers from model discrepancy since it corresponds to performing Bayesian inference for the case where the observation Y has the conditional probability density not being g(y|x) but the following:
gǫ(y|x) =
Z
Y
g(u|x)κǫ(y|u)du.
Therefore, we say that the ABC posterior is not ‘calibrated’. A way of rephrasing this is that if the model included an error term, characterised by κǫ, then the ABC would target
leads to the method of noisy ABC [Dean et al., 2011; Fearnhead and Prangle, 2012], which adds noise to the (summary statistic of) data itself to have yǫ∼ κ
ǫ(·|y), and then
perform ABC for the modified data by targeting πǫ(x, u|yǫ), which is calibrated.
Other than approximating the posterior distribution of a single random variable, the ABC approach also extends to sequential inference. Jasra et al. [2012] propose an ABC implementation scheme for approximating the densities like πn(x1:n|y1:n) in hidden
Markov models (HMM), where {Xt}t≥1 is a Markov process and the distribution of the
observables {Yt}t≥1 conditioned on the hidden process is intractable. Their approach is
related to the convolution particle filter of Campillo and Rossi [2009]. Dean et al. [2011] discuss the ABC implementation for HMMs further and show that the model for which noisy ABC is exact is also a HMM; therefore they conclude that noisy ABC can be im- plemented for HMMs. We will see the sequential implementation of ABC as well as its use for static parameter estimation in more detail in Chapter 6.
While decreasing the value of ǫ obviously makes πǫ(x, u|y) close to the true posterior,
the variance of its Monte Carlo becomes a more crucial issue. Therefore, it is important to keep the variance of the approximation at a reasonable level while making ǫ suffi- ciently small. For this reason, SMC samplers are used for approximating a sequence of distributions {πk(x, u|y) = πǫk(x, u|y)}1≤k≤p, where ǫ1 > . . . > ǫp = ǫ and the difference
between successive πk’s are small enough to make successive πk(x, u|y) varying sufficiently
slowly. SMC samplers are used in the ABC context in Sisson et al. [2007] and the method there was improved in Beaumont et al. [2009]; Toni et al. [2009] and Sisson et al. [2009]. Del Moral et al. [2012] showed the relation of these works to SMC samplers explicitly. Other novelties of Del Moral et al. [2012] is that the authors rely on M repeated simu- lations of the pseudo-data u and benefit the variance reduction property of Monte Carlo averaging and they propose a scheme for adaptive selection of the sequence of tolerance levels {ǫk}1≤k≤n. The forward kernel at step k is chosen to leave πk−1 invariant and
backward kernel is chosen to satisfy (2.19).
Other than inference of hidden variables, there is a lot of work for model selection using ABC methods. We will not review these methods as they are not of particular interest for this thesis; the interested reader may see Marin et al. [2011] for a review and the references therein for details.
Hidden Markov Models and
Parameter Estimation
Summary: This chapter contains the second half of the literature survey. The main purpose of this chapter is to introduce hidden Markov models (HMM), which are also known as general state-space models, and review their use in the literature as a powerful framework for filtering and parameter estimation.