3.3 Computational considerations
3.3.2 Conditional simulation: the Metropolis-adjusted Langevin algorithm
gorithm
The preceding discussion focused on how circulant embedding of the covariance matrix for a discretised approximation to a spatially continuous Gaussian field can result in a significant decrease in computational requirements for simulation of said field via the FFT. By “simulation of [the] field”, we refer to the fact that, given a correlation function rand values or estimates of the parametersμ,σ2andφ, we may subsequently generate a realisation
of the Gaussian variable with this specific covariance structure.
Alone, this is termed unconditional simulation, owing to the fact that the realisations of the process have been generated entirely at random i.e. without any need to take into account observed data. Clearly, if we assume the spatial or spatiotemporal intensities of a given point pattern (or at least a component thereof) is defined in terms of a LGCP, then the observed data must play a crucial role in quantifying the assumed process. Thisconditional simulation of LGCPs, hence Gaussian fields, is of course not at all straightforward due to their stochastic nature.
Conditional simulation of the LGCP is therefore also referred to asprediction: based on data we aim to predict the specific form of the stochastic intensity process instrumental in their generation. Though the conditional distribution of the stochastic process given the data is analytically intractable (Møller et al., 1998), using Bayes’ theorem it is possible to express this function up to a constant of proportionality. This motivates a Bayesian approach whereby characteristics of the posterior can be estimated through long-run averages of a suitably designed Markov chain. It is assumed the reader is familiar with the basic premise of Bayesian analysis and Markov chain Monte Carlo (MCMC); a brief introduction of the methods and terminology relevant for our purposes is given here.
At the heart of MCMC is therandom walk Metropolis-Hastings(MH)algorithm(Metropo- lis et al., 1953; Hastings, 1970). Suppose we have some objective variable of interest, V, which we wish to simulate based on the target density thereof, f. Given that the current position, or state, of the chain isV(t), we
2. Take V(t+1)= V with probability p(V , V(t)) V(t) otherwise. where p(x, y) = 1∧f(x)q[y|x;h] f(y)q[x|y;h] (3.42)
3. Repeat 1. and 2. until the desired length of the chain is reached.
Here,pis called theacceptance probability andqis referred to as thecandidate orproposal distribution. A candidate for the next transition is generated according to this distribution, which is conditional upon the current state of the chain and controlled by atuningparameter h >0. Alternatively we may write the algorithm as havingq(· |V(t);h) =V(t)+
t, where ∼q(h) andq(h) is some standardised error distribution independent ofV(t).
Thus, the MH algorithm is useful when we are able to evaluate the target densityf up to a constant of proportionality but not directly, due to the cancellation which naturally occurs in computation ofp. By subsequently obtaining multiple instances of variablesV ∼f, it is possible to therefore draw conclusions about the general posterior distribution off.
TheMetropolis-adjusted Langevin algorithm(MALA), also known as theLangevin-Hastings algorithm, was first suggested in statistics in a discussion paper by Besag (1994); a de- tailed study by Roberts and Tweedie (1997) followed. The MALA is essentially a more sophisticated random walk MH algorithm, where gradient information on the target vari- able is exploited in order to assist steering the suggested candidates toward higher areas of the target density. To denote this dependence of q on the gradient function, we write q[· |V(t);h,∇(V(t))], where∇(x) =∂log[f(x)]/∂x. This, for the MALA, takes the specific formq[· |V(t);h,∇(V(t))] =V(t)+ 0.5h∇(V(t)) +√h
t, where∼q(·). Note that the stan- dardised error distributionq for the MALA in this form does not need to account for the tuning parameter has it already scales the deterministic part of the candidate generating function. Following re-specification of q, progression of the MALA then follows the same steps as above.
Møller et al. (1998) advocate use of the MALA over a vanilla random walk for conditional simulation of spatial LGCPs due to superior convergence rates of the chains as a result of including the gradient information. This was echoed in Brix and Diggle (2001) for the spatiotemporal LGCP. Superior behaviour in terms of mixing characteristics of the MALA chain for the LGCP was also apparent in Christensen et al. (2001) and Christensen and Waagepetersen (2002). Below we summarise the quantities necessary to implement the MALA in both spatial and spatiotemporal settings. Further details are can be found in the above papers, as well as in Chapter 10 of Møller and Waagepetersen (2004).
Spatial
The variable of interest is the Gaussian random fieldY(m, n) for cell centroids (m, n)∈C as per (3.29). To take advantage of the FFT, we extend our field of interest toYext(m, n);
(m, n) ∈ Cext (3.33). Take Q to be the matrix which diagonalises ΣYext and for ease of
˜
Yext such that
˜
Yext =QΓ +μλ (3.43)
where Γ∼Nd(0,I),Nd denotes thed-dimensional standard normal distribution, andI the d-dimensional identity matrix. In addition to a slight reduction in computational complex- ity, Møller et al. (1998) note that working with Γ rather than ˜Yextdirectly when constructing the Markov chain appeared to improve its mixing, leading to an accelerated drop in autocor- relation between successively accepted draws from the target density. It is also important to note that obtaining ˜Yext from Γ via (3.43) is a relatively trivial operation when employ- ing the Fourier transforms; indeedQneed not be computed explicitly. Once ˜Yext has been
recovered, this is easily mapped back to the original lattice Iext, and stored in a relevant (Mext −1)×(Next −1) matrix. The lack of concern over edge effects means we simply
discard all cells whose centroids Cext ∈W, and we are left with an appropriate realisation of the Gaussian field, conditional upon the data, within the study region of interest.
Note that the posterior of concern is conditional upon the observed point pattern, in the sense we are interested in f(γ|X) (the conditional density of Γ|X). Møller et al. (1998) provide the relevant formula.
log [f(γ|X)] = constant +
(m,n)∈Cext
˜
yext(m, n)N(m, n)
−exp[˜yext(m, n)A(m, n)]
−0.5||γ||2, (3.44)
where ˜yext is obtained from the particular γ via (3.43), N(m, n) denotes the total number of observations falling inside the cell referenced by its centroid at (m, n), andA represents the area of each cell i.e. A= Δx×Δy as per (3.32).
Denote thed×1 vector of cell counts corresponding to the order of ˜yext by ˜Next, and
the cell area vector by ˜Aext. The gradient of (3.44) is then thed×1 vector given with
∇(γ) = ∂log[f(γ|X)]
∂γ =Q
#˜
Next−exp{y˜ext} ◦A˜ext
$
−γ; (3.45)
recall ◦ represents the Hadamard product. It is important to note that in (3.44) we set
N(m, n) =A(m, n) = 0 if (m, n)∈W; resulting in the vectors ˜Nextand ˜Aextin (3.45) being altered accordingly.
Now supposeγ(k)is the current state of the chain. For the LGCP,q=Nd(0,I), which
translates to the proposed candidate,γ, being generated as γ = Γ ∼q[Γ|γ(k);h,∇(γ(k))] =N
d(γ(k)+ 0.5h∇(γ(k)), hI). (3.46) Note that thehtherefore represents the (stationary) variance of the mutually independent
multivariate Gaussian distribution. The move fromγ(k) toγ is accepted with probability p(γ, γ(k)) = f(γ|X)q[γ (k)|γ;h,∇(γ)] f(γ(k)|X)q[γ|γ(k);h,∇(γ(k))] = f(γ|X)exp{−||γ (k)−γ −0.5h∇(γ)||2/(2h)} f(γ(k)|X)exp{−||γ −γ(k)−0.5h∇(γ(k))||2/(2h)}. (3.47)
using the definitions in (3.44) and (3.45).
To begin the chain, we must specify an initial state,γ(0). In spite of better performance
of the MALA over the random walk in terms of accelerated convergence, Christensen et al. (2003) demonstrated the apparent increased sensitivity of the MALA chain to the choice of the initial state. They concluded that, for the LGCP, electing γ(0) = Γ(0) ∼ N
d(0,I) provided the most stable performance; this author’s experience supports their findings.
Setting h is a little more difficult. However, there exists evidence in the literature to support choosing this variance of the proposal distribution to achieve an overall acceptance rate of around 0.574 to provide the fastest rates of convergence (Roberts and Rosenthal, 1998). The optimal value ofhwill differ according to various aspects of the problem such as dand chain length, and is typically set following examination of some shorter preliminary runs of the algorithm.
Should we include a multiplicative deterministic intensity ζλ in our model as described in Section 3.1.3, this changes the specific form of the target density and gradient functions. The target (3.44) becomes
log [f(γ|X)] = constant +
(m,n)∈Cext
[log{ζλ(m, n)}+ ˜y(m, n)]N(m, n)
−ζλ(m, n)exp[˜y(m, n)]A(m, n)
−0.5||γ||2, (3.48) and (3.45) becomes ∇(γ) =∂log[f(γ|X)] ∂γ =Q #˜
Next−ζ˜ext,λ◦exp{y˜ext} ◦A˜ext
$
−γ, (3.49)
where ˜ζext,λ, is the vectorised version of ζλ(m, n) with order corresponding to ˜yext. Like
N(m, n) andA(m, n),ζλ(m, n) and the relevant entries of ˜ζext,λare set to zero form, n∈W. The acceptance probability retains the same form as in (3.47).
Spatiotemporal
Dealing with a spatiotemporal LGCP MALA means we must consider multiple spatial in- tensitiesat each iteration, rather than just single realisations as above, due to the effect of the temporal correlation on the spatial variation. Under the modelling assumptions made in Section 3.1.2, Brix and Diggle (2001) simulate Γj ∼Nd(0,I) such that
˜
Zext,1=QΓ1+μψ; Z˜ext,j =QΓj+μψ[1−τ(1;θ)] +τ(1;θ) ˜Zext,j−1; j∈ {2,3, . . . , T};
recall thatτ is the temporal correlation function and unit time intervals dictate the form of the subsequentγs at timesj >1. We useZ to denote the spatially discretisedZ; ˜Zext,t is thed×1 vector giving the torus-wrapped Gaussian field on the extended spatial lattice at timet.
In disease surveillance, the goal is typically to simulate the intensity at the latest time point for which data have been observed, given data up until this time i.e. ˜Zext,T given
X1:T. However, and as Brix and Diggle (2001) are quick to point out, the nature of the model dictates it is easier to simulate ˜Zext,1:T given X1:T; owing to the fact that we only know the likelihood function up to a constant of proportionality for this situation, and not
˜
Zext,T|X1:T.
In practice, for large data sets, the simulation ofT spatial intensities at each iteration of the chain presents another computational efficiency problem. This is avoided by considering only the most recentstimes with data; a sensible solution given that we would expect the distant past to have an ever-diminishing impact on the state of the intensity at the present. LetU =T−s+ 1. We are only interested in, and hence only simulate, ˜Zext,U:T givenXU:T. The value ofsis chosen according to the strength of temporal correlation for the application at hand, which we recall is controlled byθ. We will think along these lines for the remainder of the definitions, which means the simulation design in (3.50) changes with Γ1 = ΓU and
j∈ {U+ 1, U+ 2, . . . , T}.
The log-target in the spatiotemporal setting is given by Brix and Diggle (2001) as
log[f(γU:T|XU:T)] = constant + T t=U (m,n)∈Cext ˜ yext,t(m, n)Nt(m, n)
−exp[˜yext,t(m, n)]A(m, n)
− T t=U+1 ||γt||2 2[1−τ(1;θ)2] −0.5||γU||2, (3.51)
where Nt(m, n) denotes the number of observations falling in the cell with centroid (m, n) at timet.
As earlier, let ˜Next,trepresent the d×1 vector of cell counts at timet with order corre- sponding to ˜yext. We express the gradient as thed×sarray with
∇(γU:T) =
∂
∂γUlogf(γU:T|XU:T)
, . . . , ∂ ∂γTlogf(γU:T|XU:T) , where ∂ ∂γt logf(γU:T|XU:T) =1[t < T] T−1 i=t
Q#τ(1;θ)( ˜Next,T+t−i−exp[˜yext,T+t−i]◦A˜ext)
$[T−i]
−[1−τ(1;θ)2]−1γt, (3.52)
and using similar notation as for (3.40),abhere refers to the elementwise power of the vector ato the scalarb. In both (3.51) and (3.52), the ˜yext,ts are obtained with the corresponding γts via (3.50), and the terms involving (m, n) are again set to zero for those (m, n) ∈W.
Conceptually, it is easiest to think of∇(γU:T) as some list object withs components, each a vector of lengthddescribing the gradient at each spatial cell for each of the timestamps
{U, . . . , T}.
Candidates for each transition in the MALA are generated in much the same way as in the purely spatial case. They are now treated as ‘blocks’ ofsconsecutive spatial intensities such that the candidateγU:T can be viewed as ad×s matrix, with each column corresponding to a single spatial intensity onIext. AssumingγU(k:)T is the current state of the chain, and that the gradient term is an appropriately constructedd×smatrix, we have
γU:T = ΓU:T ∼Nd(γU(k:)T+ 0.5h∇(γ
(k)
U:T), hI). (3.53) The moveγU(k:)T →γU:T is accepted with the now familiar probability
p(γU:T, γU(k:)T) = f(γU:T|XU:T)exp{−||γ (k) U:T −γU:T−0.5h∇(γU:T)||2/(2h)} f(γU(k:)T|XU:T)exp{−||γU:T −γ (k) U:T−0.5h∇(γ (k) U:T)||2/(2h)} . (3.54)
An initial state follows the same guidelines mentioned earlier, in the sense that γU(0):T is given with γi(0) ∼ Nd(0,I) for i ∈ {U, . . . , T}. The tuning parameter h in subsequent candidate generation is again chosen to achieve an overall approximate acceptance rate of 0.574.
The inclusion of deterministic components describing ‘global’ spatial and/or temporal trends, ζψ and η as in Section 3.1.3, alters the target density and gradient as we would expect, with (3.51) becoming
log[f(γU:T|XU:T)] = constant + T t=U (m,n)∈Cext {y˜ext,t(m, n) + log[ζψ(m, n)η(t)]}Nt(m, n)
−ζψ(m, n)η(t)exp[˜yext,t(m, n)]A(m, n)
− T j=U+1 ||γj||2 2[1−τ(1;θ)2] −0.5||γU||2, (3.55)
and the entries for timet in (3.52) becoming ∂ ∂γtlogf(γU:T|XU:T) =1[t < T] T−1 i=t Q#τ(1;θ)N˜ext,T+t−i
−η(T+t−i)˜ζext,ψ◦exp[˜yext,T+t−i]◦A˜ext
$[T−i]
−[1−τ(1;θ)2]−1γt, (3.56) where, as in the purely spatial setting, ˜ζext,ψ is the corresponding vector of deterministic spatial values overIext.
A noteworthy comment, pertaining to both purely spatial and spatiotemporal mod- els, concerns gradient truncation. The MALA can, in some cases, spend lengthy periods away from the modes of the target density which can in turn adversely affect the mixing
and convergence of the chain, and therefore our final results (Roberts and Tweedie, 1997; Møller et al., 1998). This can be avoided, without disturbing the ergodic properties of the chain, by imposing a constraint on the magnitude of the cell-wise gradient values. The truncated MALA is obtained by replacing exp[˜yext,t] in (3.45), (3.49), (3.52) and (3.56) by
{a∧◦ exp[˜yext,t]}, where a represents the d×1 vector with all entries equal to the scalar
constant a, and∧◦ denotes the component-wise minimum. At the time of writing there ap- pears to be no research concerning ‘optimal’ calculation ofa, though this is not necessarily a major concern in practice. As mentioned in Møller et al. (1998), truncation is generally not needed provided sensible specification of the tuning parameterh; typically achieved if we search for the optimal acceptance rate of 0.574.
3.4
Real-world examples
We now take a break from the theory and demonstrate epidemiological modelling with the LGCP using two real-world data sets; one purely spatial, the other spatiotemporal. The examples are here mainly for illustrative purposes, though we do provide conjecture on possible conclusions. Relative to the existing literature there are novel aspects to both analyses in terms of the data used and methods employed.