Gibbs sampler

Our Gibbs sampler acts on augmented state-space _{λ,Q, Xt}, and each iteration

has 3 distinct stages:

1. Given the parameter values (λ,Q) use the second form of the the forward- backward algorithm, specified by (2.2) and (2.3) in Section 2.2), to simulate the state of the hidden chain Xt at the start and end of the observation

interval (t′

0 = 0 and tobs = t′n+1) and at a set of time points t′1, . . . , t′n. For

data format D1 t′

1, . . . , t′n correspond to event times; for formats D2 and D3

t′

1, . . . , t′n+1 are the end-points of accumulation intervals.

2. Given the parameter values and the finite set of states produced in stage 1, apply the technique of Section 2.3 to each interval in turn to simulate the full underlying hidden chainXt from it’s exact conditional distribution.

3. Simulate a new set of parameter values.

We now describe how each of the stages may be implemented for each of the three data formats.

Data format D1

For stage 1 we apply the forward-backward algorithm of section 2.2 modified to take account of the fact that observation times t′

1, . . . , t′n correspond exactly to

events of the observed process and that therefore there are no Y-events between observation times. For the kth _{interval, which has width t}

k =t′k−t′k−1, the tran-

point isl(k) _=λ.

This process is exactly equivalent to straightforward application of the second form of the forward-backward algorithm to the meta-process Wt of section 2.4.1 on the

extended state space _{1, . . . , d,1∗_{}, but replacing thed-dimensional vector 1 with}

the d+ 1-dimensional vector (1, . . . ,1,0)t_{. For the k}th _{interval, the transition ma-}

trix is now T(k) _=eGwtk, where G

w is defined in (2.7) and eGwt is given explicitly

in (2.8). The likelihood vector isl(k)_{= (λ,0)}t_.

Stage 2 applies the technique of Section 2.3 directly to extended state space {1, . . . , d,1∗_{} with generator matrix G}

w.

Figure 2.1 shows the first two stages for data formatD1.

Stage 3 is especially simple if conjugate gamma priors are used for the parame- ters since the likelihood for the full data (observed data and complete underlying Markov chain) is L(xt,t|Q,λ)∝νs0 × d Y i=1 Y j6=i qrij ij e−qij ˜ ti × d Y i=1 λni sie −λi˜ti _(2.12)

Thus independent priorsλi ∼Gam(αi, βi) produce independent posteriors

λi ∼Gam(αi+ni, βi+ ˜ti) (2.13)

Were it not for the factorνs0, which is itself a function ofQ, choosing independent

priors qij ∼Gam(γij, δij) (j 6=i) would lead to independent posteriors

CHAPTER 2. BAYESIAN ANALYSIS OF THE MMPP ₄₉

2 1

1

2

2

1

1

2

1

1

Figure 2.1: The Gibbs sampler (a) first simulates the chain state at observation times and the start and end time; for each interval it then simulates (b) the number of dom- inating events and their positions, and finally (c) the state changes that may or may not occur at these dominating events. The figure applies to a two-state chain with

λ2+q21> λ1+q12 .

However since νs0 is bounded between 0 and 1 we may employ rejection sampling,

simulating Q from (2.14) and accepting with probability νs0(Q).

Data format D2

For stage 1 we apply the second form of the forward-backward algorithm with likelihood vector l(k) _{=1and transition matrix dependent on the binary indicator}

(bk) for the interval

T(k) =P(0)1−bk Pbk

Forstage 2 we first consider the meta-processWton state space{1, . . . , d,1∗, . . . , d∗}

This has generator matrix Gw=   Q₋Λ Λ 0∗ _Q  

For a given interval suppose that we have simulated Xt starting in state s0 and

ending states1. On the extended state space this corresponds to starting in state

s0 and finishing in states1 if there have been no events over the interval, otherwise

finishing in s∗

1 . We simulate the underlying chain from the algorithm of section

2.3. This also supplies the time of the first event in the interval, which we use for simulating the new parameters in stage 3.

In stage 3, for accumulation interval i define t∗

ij as the amount of time that the

hidden chain spends in state j between the start of the interval and either the time of the first event (if there is a first event) or the end of the interval; write t∗

·j =

P

it∗ij. Let n∗j be the number of intervals for which the chain is in statej at

the first event of the interval. Then the likelihood is L(xt,t|Q,λ)∝νs0 × d Y i=1 Y j6=i qrij ij e−qij ˜ ti_× d Y j=1 λn∗j j e−λjt ∗ ·j

We then proceed as with data format D1.

Data format D3

For this data format we consider the meta-process Vt on extended state space

{1(0)_{, . . . , d}(0)_,1(1)_{, . . . d}(1)_{, . . . ,1}(cmax)_{, . . . , d}(cmax)_,1∗_{} as defined in section 2.4.2.}

For the application of the forward-backward algorithm in stage 1, the transition matrices are T(k) _{= P}(ck) and the likelihood vectors are l(k) = 1. For stage 2, in

CHAPTER 2. BAYESIAN ANALYSIS OF THE MMPP ₅₁

simulating from the exact distribution of the underlying chain for an interval where the start state iss0, the end state is s1 and there areck events observed we use the

generator matrix Gv as defined in (2.11) with start state s0 but end state s(1ck).

The algorithm also simulates from the exact distribution of the times at which each of theck events occurs over the interval, therefore we may performstage 3 exactly

as for data format D1.