Background

6.2.1

Simulated Annealing

Assume that hcan be evaluated point-wise. In classical simulated an- nealing (

sa

) (Kirkpatrick, Gelatt & Vecchi, 1983), the target measures

psa 2M.ΘZp/ are defined according to Equation 6.2 with Zp ;, Hp. /Dexp.ˇph. //. That is, the target measures of

sa

are defined by

_psa.d /_WDexp.ˇph. //.d /:

In the particular case that D Leb˝d is the Lebesgue measure on ΘDRd,his three times continuously differentiable, and under further

technical conditions on Θh, Hwang (1980) showed that this family of

distributions concentrates aroundΘhas formalised in Equation 6.1. The

idea is then to approximate the distributions.psa/p2N, wherepsa /psa,

using an inhomogeneous

mcmc

algorithm, i.e. using an

mcmc

chain whose target distribution changes over iterations. Under strong regularity conditions, converge of

sa

was established in Hajek (1988), Winkler (2003), Andrieu, Breyer and Doucet (2001).

Finally, Rubenthaler, Rydén and Wiktorsson (2009) studied improve- ments upon the annealing schedule and target measures in

sa

.

6.2.2

State Augmentation for Marginal Estimation

Motivation. Algorithms such as

sa

require point-wise evaluations of

(maximum-preserving transformations of) h. The rest of this work is

concerned with situations in which such evaluations are impossible (to do efficiently) but in which we can find some spaceZp, some finite kernel Mp, and some measurable functionHp (which wecanevaluate) such that

the measuresp satisfy Equation 6.1.

More specifically, we assume that there exists a measurable function

HW ΘX!Œ0;1/and a stochastic kernelM 2 K₁.ΘX/such that .;dx/ WDH.; x/M.;dx/

defines a finite kernel and such that – recalling that1is the unit function

on an appropriate domain – the maxima of the integral

.;1/WD Z

6.2 Background coincide with the maxima ofh, i.e.

ΘhD

˚

02Θˇˇ8 2 ΘW.0;1/.;1/ : (6.4)

For later use, we define the stochastic kernel˘ˇ by ˘ˇ.;dx/WD

H.; x/ˇ_M.;dx/

R

XH.; x/

ˇ_M.;dx/: (6.5)

6.2 Example (

ml

/

map

estimation, continued). Many statistical mod- els are specified through an additional set of latent variablesXtaking values in some setX. Under such a model, conditional on some 2 _Θ,X is dis- tributed according toM.; /. LetG.; x/denote the completed likelihood, i.e. the likelihood of some observed data given the parameters and latent variables,.; x/. Unfortunately, the marginal likelihood

L. /D Z

X

G.; x/M.;dx/

is often intractable. To still perform optimisation in this setting, we can take

(1) H.; x/ WDG.; x/for the purpose of marginal

ml

estimation, (2) H.; x/ WDG.; x/p. /Q for the purpose of marginal

map

estimation.

A

mcmc

algorithm for performing optimisation in such problems called

state augmentation for marginal estimation(

same

) was introduced in an

mcmc

context by Doucet et al. (2002) (see also Gaetan & Yao, 2003; Jacquier, Johannes & Polson, 2007). Here, we describe the slightly more general construction developed by Johansen et al. (2008) which allows for non-integer inverse temperaturesˇp. Again,.ˇp/p2N is a sequence inŒ0;1/such thatˇp " 1.

Extended Target Measure. At thepth iteration, the

same

algorithm

augments the space withdˇpereplicas of the random variableX DXp,

i.e.Zp WD X1Wdˇpe. The extended target distributionspsame /psame on

ΘZp withZp WDXdˇpe, are then defined by lettingpsame be given by

Equation 6.2 with Mp.;dzp/WD dˇpe Y iD1 M.;dx_pi/;

and with Hp.; zp/WDH.; xd ˇpe p /ˇ ] p bˇpc Y iD1 H.; x_pi/; whereˇ] WDˇ bˇc. Wheneverˇp 2 N, Z AZp _psame.d dzp/D Z A .;1/ˇp_.d _/;

so that under suitable regularity conditions, by Equation 6.4, the -

marginal ofpsameconcentrates aroundΘhasp ! 1.

Algorithm. The

same

algorithm is usually implemented as an inhomo-

geneous Gibbs sampler or inhomogeneous Metropolis-within-Gibbs al- gorithm as summarised in Algorithm 6.3. Therein, Qp denotes some

mcmc

kernel which is invariant with respect to the full conditional dis- tribution ofunder_psame andRp is an

mcmc

kernel targeting

(

˘˝dˇpe

1 .; /; ifˇp 2N, ˘˝bˇpc

1 ˝˘ˇp].; /; otherwise,

where˘ˇ was defined in Equation 6.5. Furthermore, Sp 2 K1 ΘXdˇp 1e;Xdˇpe dˇp 1e

is some suitable proposal kernel for augmenting the space with additional latent variablesXdˇp ₁eC1Wdˇpe

p ₁ , wheneverdˇpe>dˇp ₁e. To simplify the

notation, we write the thus augmented vector of latent variables as

z Zp ₁WDX1Wd ˇpe p ₁ D Zp ₁; Xd ˇp 1eC1Wdˇpe p ₁ :

This random vector takes values in the setZzp 1WDXdˇpe.

6.3 Algorithm (

same

). At thenth iteration, (1) ifdˇ_ne>dˇ_{n 1}e, sampleX_ndˇn 1eC1Wdˇne

1 Sn..n ₁; zn ₁/;/, (2) sampleZ_nDX_n1WdˇneR_n.._{n 1};zQ_{n 1}/; /,

6.2 Background

Potential Drawbacks. Usually, the

same

algorithm is implemented as

an inhomogeneous Gibbs sampler or Metropolis-within-Gibbs algorithm. This can lead to poor mixing of the

mcmc

chain for two reasons.

(1) It is often impossible to updateX1Wdˇpe

p using a Gibbs step and devis-

ing another efficient form for the

mcmc

kernelRp can be difficult,

especially ifXis high-dimensional.

(2) IfandZp are highly correlated underpsame, then component-wise

updates are known to be inefficient.

To alleviate the first potential drawback, we propose to combine the

same

approach with sophisticated multiple-proposal

mcmc

kernels such as (iterated)

csmc

kernels (Andrieu et al., 2010; Whiteley, 2010). This is described in the next section. To alleviate the second potential drawback, we also present a sequence of further extended distributions. This permits the combination of the

same

approach with pseudo-marginal

mcmc

kernels (Beaumont, 2003; Andrieu & Roberts, 2009).

6.2.3

Optimisation Using

smc

Samplers

Even if the objective functionh(and also.;1/) is unimodal, the intro-

duction of the latent variablesX1Wdˇpe

p often leads to an extended measure _psamewith varying degrees of multimodality. This can hamper mixing of

the inhomogeneous

mcmc

algorithms targeting the distributionspsame.

Furthermore, ifΘhcontains multiple maxima, a single

same

chain will

eventually become trapped around one of them.

To improve robustness, Johansen et al. (2008) devise a population-based version of the

same

idea. More precisely, they incorporate the

same

approach into the

smc

-sampler framework from Del Moral et al. (2006b, 2007), Peters (2005) which is summarised in Subsection 2.3.2. Such

smc

samplers have already been successfully employed to realistic problems in air-traffic management (Kantas, Maciejowski & Lecchini-Visintini, 2009, 2010).

At thetth step, the

smc

sampler uses (forward) proposal kernels which

move the particles according to an

mcmc

kernelPzt, given by

z

Pt..t ₁;zQt ₁/;dt dzt/

This

mcmc

kernel is applied after having extended each particle via

St 2 K₁.Θ Zt ₁;Xdˇte dˇt 1e/, in the case that dˇte > dˇt ₁e. In

summary, recalling thatZzp 1DXp1Wdˇ₁pe, the Step-t proposal kernel is Pt..t ₁; zt ₁/;dxdtˇt₁ 1eC1Wdˇtedt dzt/

WDSt.t ₁; zt ₁/;dxd

ˇt 1eC1Wdˇte

t ₁ /

zPt..t ₁;zQt ₁/;dt dzt/:

As stressed in Johansen et al. (2008), this is not the most general

smc

implementation of the

same

idea; letting_Pz_{t be an}

_mcmc

_{kernel is often}

convenient but not actually necessary.

The

smc

sampler marginally targets the measuretsame by targeting

an extended measure constructed via backward Markov kernels,

Lt ₁..t; zt/;dt ₁dzQt ₁/

WD Pzt..t_d 1;zQt 1/; / tsame

.t; zt/tsame.dt ₁dzQt ₁/:

In other words, this is the usual approximation to the optimal backward kernel described in Example 2.15. This backward kernel is commonly employed whenever

mcmc

kernels are used to move the particles within an

smc

sampler.

With these backward kernels, and withUt WD.t; Zt; Xtdˇ₁t 1eC1Wdˇte/,

the incremental importance weights at Stept are defined by Gt.u₁Wt/WD Q

t

Q

t 1˝St

.t ₁;zQt ₁/:

This may also be derived by viewing the single step of the

smc

sampler as two consecutive

smc

steps. The first step then changes the extended target distribution (and potentially adds additional variables using the proposal kernelSt), leading to the above-mentioned incremental weight.

The second step simply applies the

mcmc

kernel_Pz_{t to the particles using}

time-reversal backward kernels. As noted in Example 1.9, this second step does not modify the weights.

In document On extended state space constructions for monte carlo methods (Page 191-196)