Simulation of Markov Processes

The main task in this thesis is to devise simulation algorithms for point process models via the construction of Markov processes whose equilibrium distribution is exactly that of the point process of interest. A natural state space for such processes will be the exponential spaceXe, for individuals

lying in some spaceX. The type of Markov processes dealt with here will either be discrete-time ‘component’ processes or spatial birth-death processes (Preston 1977). A component process with

d-componentsΦ = (Φ1, . . . ,Φd)is just ad-dimensional process. A spatial birth-death process is a

special kind of continuous-time Markovjumpprocess.

Definition 1.29. A stochastic process on(X,B(X))is called a jumpprocess withintensityα and transition kernelK if, given that the process is currently in stateξ∈ X, then the waiting time till the next jump has an exponential distribution with rateα(ξ), independent of the past, and the probability that the jump leads to a state inA∈ B(X)isK(ξ, A).

The intensity and transition kernel of the birth-death process will be defined viabirthanddeathrates. Furthermore if the birth and death rates of the spatial birth-death process satisfy detailed balance for some integrable function f (with respect to a measureµon (Xe,B(Xe))) then the equilibrium

distribution of the process is given byπ =R f dµ(see Section 1.3.2 for details). This means that if the distribution of a point process admits a density then it is possible to define birth and death rates such that detailed balance is always satisfied; see Example 1.3.

The dynamics of component process will be defined via one step transition probabilities or ker- nels for each component. These will be chosen in order to ensure that the equilibrium distribution of the process is exactly that of interest. One way to guarantee this is by updating each component

according to the (equilibrium) conditional distribution of that component given the other compo- nents. This is the idea embodyingGibbs sampling described in Section 1.4. For discrete-time or continuous-time jump processes one can define the notion of atransition function, which essentially determines the next state of the process.

Definition 1.30. Atransition functionfor a Markov kernelPis a measurable functionf :X × · → X, where·is the state space of some (auxiliary) random variableU, such that the lawL(f(ξ, U)) =

P(ξ,·)for allξ∈ X. IfX admits a partial order4then the transition functionf is

monotone: if f(η, U)₄f(ξ, U), wheneverη₄ξ; (1.23)

anti-monotone: if f(η, U)_<f(ξ, U), wheneverη₄ξ. (1.24)

Remark 1.1. If P has a monotone transition function then P is monotone; this is referred to as

realizablemonotonicity by Fill & Machida (2001), who also show that the converse is not true. Thus if P has equilibrium distribution π then a transition rule is simply a measurable mapping which preserves the distribution, ie. ifξ ∼ π thenf(ξ,·) ∼ π. So any mapping which preserves the equilibrium distribution is a transition function. Therefore simulation of a discrete-time Markov process Φreduces to sampling its transition function: initialize Φ (0) at some arbitrary state; for

n ≥ 1set Φ (n) = f(Φ (n−1),·). A similar set up works for jump processes, but with updates occurring at ‘jump’ times.

SupposeΨis thetargetprocess of interest which we wish to sample, but for which direct sam- pling is generally difficult. Suppose also that its transition functionf0 can be obtained as an adapted functional off, the transition function of some easy-to-simulate processΦ. In this caseΨcan be simulated bycoupling its evolutionto that ofΦ, ie. one can produce a couplingΦb,Ψb

. The transi- tion functionf is used to updateΦb and, conditional on such an update, theΨb component is updated according tof0. The processΦbis usually referred to as thedominatingorbasicorfreeprocess since it is the underlying process upon which the transitions ofΨb depend.

Example 1.2. A simple example of such a coupling is whenΨis a jump process on someW ⊆ X

with intensityα0(x) =R b0(ξ, x)µ(dξ)+D(x), whereµis some measure onX. IfR b0(ξ, x)µ(dξ)

is difficult to compute butb0(ξ, x)≤b, for some constantb, thenα(x) = bµ(W) +D(x)≥α0(x). Defining Φ as a jump process with intensity α(·) then enables one to simulate Ψ by devising a

couplingΦb,Ψb

as follows. The processes are initialized at the same state; suppose that the current state ofΦbisx. The next jump time ofΦbis simulated as an exponentially distributed random variable of rateα(x) where bµ(W), respectively D(x), represent the total birth, respectively death, rates. With probability bµ_α(₍W_x)) a birthξis proposed, drawn uniformly onW since the birth rate is constant; else a deathη∈xis proposed. HenceΦbhas transition function

f(x, U) =    x∪ {ξ} ξ∈W, ξ /∈x, ifU ≤ bµ_α(₍W_x)); x\ {η} η∈x, else.

So, conditional on a transition inΦb, the same transition is considered as aproposedtransition inΨb. Death transitions are always accepted inΨ, whereas births are accepted with probability b0(ξ,x_b ). The transition function f0 ofΨb then looks like f above but with bµ(X) replaced by

R

b0(ξ, x)µ(dξ); henceΨb has the same transition rates asΨ, as required.