Numerical Methods – Gillespie’s Algorithm

tribution P(n, t) can rarely be solved exactly. Gillespie’s algorithm is a method by which an individual sample path, starting at a given initial point, can be simulated in time such that it conforms to the unknown probability distribution we seek; that is, for a sufficiently large popula- tion of sample paths, the inferred probability distribution is as near to the exact solution as we wish (analogous with Langevin’s modeling of Brownian motion). The algorithm proceeds in 3 steps:

1. The propensitiesνj are used to generate a probability distribution

for the next reactiontime,τ andτ is drawn from this distribution. 2. The propensities are used to generate a probability distribution for

which reaction in the network will occur next,i.e. which of thevj’s

is completed at timet+τ. Call the reaction indexµ.

3. The time is advancedt→t+τ and the state is updated using the stoichiometry matrix - for each reactant,ni→ni+Siµ. Repeat. . .

In this way, we generate a discrete time series for the reactant numbers. We shall go over each of these steps in greater detail (see Figure 4.3), and it will be clear that in developing his algorithm, Gillespie built upon some fundamental properties of stochastic processes. It is also instructive to see Gillespie’s algorithm in action, so we generate some stochastic simulations of the Brusselator model introduced on page 88.

Details of the stochastic simulation algorithm (Figure 4.3)

• D. T. Gillespie (1977) “Exact simulation of coupled chemical reactions,”Journal of Chemical Physics81: 2340.

• Markov Processes,D. T. Gillespie (Academic Press, Inc, 1992). To advance the state with each reaction event, we need two random variables – the time for the completion of the next reaction τ, and the index of the reaction that firesµ. It is Gillespie’s deep insight that allows us to determine the probability distribution for each of these, and in fact how to generate the pair (τ, µ) using a unit uniform random number generator.

1. Initialize:t _←t₀, n_←n₀.

2. Pick τaccording to the density functiona,b

3.Pick µaccording to the density functiona

4.Advance the process:

•n_i←n_i+ S_iµ • t _← t + τ.c

5. Record as required for sampling or plotting.

If the process is to continue, then return to 2; otherwise, stop.

(

) ( )

( )

1 | , exp . p τ nt =a n _−a nτ_

(

)

_{( )}( )

2 | , p t a µ µ

ν

∆n n = n n

( )

, for

,

,

for

;

t

t

t

t

t

t

µ

τ

′

−

− < <



′ =

′ =



n S

n

n

Figure 4.3: Gillespie’s algorithm for stochastic simulation of the master equation. a) Use the inversion method for generating the random number τ (see Section A.4). If a(n, t) ≡ a(n), then the in- version is easy: Draw a unit uniform random number r1, and take

τ = [1/a(n)] ln(1/r1). b) Draw a unit uniform random number r2 and

takeµto be thesmallest integer for which the sum overνj(n)/a(n) from

j = 1 toj=µexceedsr2. Notice the jump in the state ∆nµ=Sµ, where

Sµ is theµthcolumn of the stoichiometry matrix. c)In a simulation run

containing∼10K jumps, the sumt+τ should be computed with at least K+ 3 digits of precision. Taken from Gillespie (1992), p. 331.

p(n+ ∆n, t+τ|n, t)dτ: the probability that, given the sys- tem is in statenat timet, the next jump occurs betweent+τ andt+τ+dτ, carrying the state fromnton+ ∆n,

from which we draw the random numbers (τ,∆n) to advance the system. In practice, because the stoichiometry matrix S records how the state advances with each reaction event, it is sufficient that we simply generate the random variable specifying which reaction has fired, (τ, µ).

Following Gillespie, we introduce the probability q(n, t;τ) that the system in staten(t)will jump at some instant betweentandt+τ. From the microscopic transition ratesν(n), we know that over aninfinitesimal

intervaldt,

q(n, t;dt)≡

Prob. Reaction 1 Occurs + Prob. Reaction 2 Occurs + . . .

= [ν1(n)dt+ν2(n)dt . . .] =   N X j=1 νj(n)  dt≡a(n)dt (4.28)

Over an infinitesimal interval, at most one jump can occur, so we have that the probability that no jump occurs, q?_{(n, t;dτ), is q}?_{(n, t;dτ) =}

1−q(n, t;dτ). Over a non-infinitesimal interval, the probability thatno

jump occurs is,

q?(n, t;τ) = exp [−a(n)τ], (4.29)

(Exercise 6). In that way, the probabilityp(n+ ∆n, t+τ|n, t)dτ is writ- ten,

p(n+ ∆n, t+τ|n, t) = q?(n, t;τ)

| {z }

Probability the state will NOT jump during [t,t +τ]

× a(n)dτ

| {z }

Probability the state WILL jump in [t +τ,t +τ+ dτ]

× w(∆n|n, t+τ)

| {z }

Probability that, given the state jumps at t +τ,

it will land inn+ ∆n

The first two terms on the right-hand side determine the next reaction

timeτ, while the last term determines the next reactionindex. Therefore, we can factor the conditional probability into two partsp1(τ) andp2(∆n),

p(n+ ∆n, t+τ|n, t) =a(n) exp [−a(n)τ]dτ | {z } p1(τ|n,t) ×w(∆n|n, t+τ) | {z } p2(∆n|n,t+τ) .

Probability p1(τ) is exponentially distributed, consequently a unit uni-

form random numberr1 can be used to simulateτ via the inversion (see

Section A.4),

τ = 1

a(n) ln(1/r1). (4.30)

The index of the next reaction µ is a little more subtle – Notice from Eq. 4.28 that the probability of reaction µ to occur is proportional to the rate of reaction νµ. The normalization condition ensures that the

probability it is reactionµthat has caused the state to jump is, p2(∆nµ) = νµ(n) P jνj(n) ≡ νµ(n) a(n), (4.31)

where ∆nµ is theµth column of the stoichiometry matrix S. The next

reaction indexµis simulated using a unit uniform random numberr2via

the integer inversion method (see Exercise 4b on p. 279). That is, the index µdrawn fromp2is the first integer for which

1 a(n) µ X j=1 νj(n)> r2. (4.32)

With (τ, µ), the system is updated, t→t+τ, ni→ni+Siµ,

and the algorithm is repeated as long as desired, each time drawing two random numbers (r1, r2) from a unit uniform random number generator.

An implementation of Gillespie’s algorithm coded in Matlab is annotated in the Appendix (see Section D.1.2 on p. 306).

Stochastic simulation of the Brusselator model

We shall produce stochastic simulations of the Brusselator model in both the stable and limit-cycle regimes of the model parameter space (see Sec- tion D.1.3 on p. 308 for example Matlab code). We shall find that both regimes are equally accessible to the stochastic simulation algorithm. In contrast, the linear noise approximation described in the next section re- quires a major modification to treat fluctuations around the limit cycle. On the other hand, it is very difficult to describe the simulation results

Number of X1molecules (x103) 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 Number of X1molecules (x103) N u m b e r o f X2 m o le c u le s (x 1 0 3) N u m b e r o f X2 m o le c u le s (x 1 0 3) A B

Figure 4.4: Gillespie’s stochastic simulation algorithm - The Brus- selator in the stable regime. a) (a, b) = (0.1,0.2), (α= 10 in Figure 23 of Gillespie (1977)). The system is very stable. b) (a, b) = (0.5,1), (α= 2 in Figure 22 of Gillespie (1977)). The system is still stable, but fluctuations are more appreciable.

any more than qualitatively, while the perturbation methods provide a connection between system behaviour and model parameters.

Stable regime,b <1 +a: In the stable regime of the model parame- ter space, the system admits a single, stable equilibrium point - In Figure 4.4, that equilibrium is (x1, x2) = (1000,2000). The number of reactant

molecules is large, so the intrinsic fluctuations are correspondingly small. As the parameters get closer to the Hopf bifurcation, the fluctuations be- come somewhat larger (Figure 4.4b). Neither plot illustrates particularly interesting behaviour.

Limit cycle, b≥1 +a: At the bifurcation (Figure 4.5), the system parameters are on the threshold of stability and the fluctuations carry the state on long excursions away from the fixed point. From the time- series plot (Figure 4.5a), it appears as if the fluctuations are generating nearly regular oscillations (see Excercise 5 on p. 214). In phase-space (Figure 4.5b), the system seems confined to an elongated ellipse with a negatively-sloped major axis.

Beyond the Hopf bifurcation, the system exhibits regular limit cycle oscillations (Figure 4.6). As Gillespie notes in his original article, the horizontal leg of the cycle seems to travel along grooves of negative slope, rather than straight from right to left. There is some spread along the diagonal leg, but both the horizontal and vertical legs are little influenced by the fluctuations (for analytic insight into why that is, see Section 11.2).

Number of X1molecules (x103) 1 2 3 4 5 1 2 3 4 5 N u m b e r o f X2 m o le c u le s (x 1 0 3) B 1 2 3 4 5 N u m b e r o f Xj m o le c u le s (x 1 0 3) A 1 2 3 4 5 6 7 8 9 10 Time

Figure 4.5: Gillespie’s stochastic simulation algorithm - The Brus- selator at the Hopf bifurcation. a) Simulation of the systemat the Hopf bifurcation, (a, b) = (1,2) (α= 1 in Figure 21 of Gillespie (1977)). The fluctuations generate what appear to be almost regular oscillations.

b) In phase-space, the fluctuations are confined to a large ellipse, an- gled with negative slope indicating the strong cross-correlation between fluctuations inx1 andx2. Number of X1molecules (x103) 6 8 10 12 14 4 6 8 12 14 Number of X1molecules (x103) N u m b e r o f X2 m o le c u le s (x 1 0 3) N u m b e r o f X2 m o le c u le s (x 1 0 3) A B 10 2 4 6 8 12 14 10 2 2 4 2 4 6 8 10 12 14

Figure 4.6: Gillespie’s stochastic simulation algorithm - The Brus- selator in the unstable regime. a) (a, b) = (5,10) (α= 0.2 in Figure 19 of Gillespie (1977)). b) (a, b) = (10,20) (α = 0.1 in Figure 18 of Gillespie (1977)).

The advantages of the stochastic simulation algorithm is that it is sim- ple to program and provides an output trajectory that exactly conforms to the solution distribution of the master equation. The disadvantages are that the original algorithm is computationally expensive and the method

does not scale well as the number of molecules gets large (although there are approximate algorithms that alleviate some of the computational bur- den). Most importantly, the method suffers from the same limitations as any numerical scheme – there is a lack of deep insight into the model and it is difficult to systematically explore different regions of parameter space. Nevertheless, Gillespie’s algorithm is the benchmark against which all other methods of solving the master equation are measured.

Suggested References

For numerical simulation methods, the textbook by Gillespie is unsur- passed, along with his seminal article on the stochastic simulation algo- rithm,

• D. T. Gillespie,Markov processes: An introduction for physical sci- entists (Academic Press, 1992).

• D. T. Gillespie (1977) “Exact simulation of coupled chemical reac- tions,”Journal of Chemical Physics 81: 2340.

Exercises

1. Separable transition matrices: Write out the most general 2×2 transition matrix _W. Show that for this case, any initial con- dition p(0) converges to a well-defined steady-state distribution

ps_{,i.e. lim} t→∞W

t_{·p(0) =p}s_{, with two exceptions.}

2. Time-dependent reaction rates: A chemical reaction is de- scribed by the following deterministic rate equation,

dA

dt =k(t)A(t),

wherek(t) is a time-dependent reaction rate. Solve the associated chemical master equation using a moment generating function. 3. Bursty Poisson model: Consider the following generalization of

the Poisson model: synthesis occurs with a constant rate α, but in ‘bursts’ of size b, and degradation is linear, with rate βn. The

master equation corresponding to this process is, dP(n, t)

dt =

α[P(n−b, t)−P(n, t)] +β[(n+ 1)P(n+ 1, t)−nP(n, t)]. (a) Forb= 1, solve the characteristic functionQ(z, t) for all time.

What do you notice about the distribution ifn0= 0?

(b) Repeat 3a, but for arbitrary burst size.

4. Bernoulli’s urn model:In describing Bernoulli’s urn model, Laplace derived a difference equation with nonlinear transition rates (Eq. 4.20). The nonlinearity of the transition rates make the equation difficult to solve even at steady-state.

(a) Using matrix iteration for the casesn = 2,3, and 4, multiply the steady-state probability distributionzby its first element z1 to get a vector of integers. Can you spot a pattern in the

individual entries? Postulate a general solution for the steady- state probability distribution and verify that it satisfies the difference equation.

(b) Using Laplace’s approximate solution (Eq. 4.22), calculate the mean-squared error between the exact solution and the contin- uous approximation as a function of the number of balls. 5. Asymmetric cell division: Most bacteria divide with surprising

symmetry –E. colifor example, typically divides into daughter cells that differ in length by less than 5%. Suppose a bacterium divided

unequally, what would the age distribution (i.e., the time to next division) look like for a population?

(a) Divide the lifetime of the cell into time-steps ∆t– Assume the larger daughter lives 10∆t before division, while the smaller daughter lives 15∆t. Scale the time so that the transition rates are 1. Write out the transition matrix_W.

(b) Find_Ws_{. What does the equilibrium lifetime distribution look}

like?

(c) Repeat the above, but on a finer scale. That is, assume the large daughter lives 100∆tand the small daughter lives 150∆t. (d) Derive a deterministic model for the process. How do the re-

(e) Suppose the size after division was itself a stochastic process. How would the transition matrix change?

6. Gillespie’s simulation algorithm:Gillespie draws upon two very important ideas in stochastic processes – the evolution of probability for a Markov process, and the simulation of a random variable by a unit uniform random number.

(a) Making use of the Markov property, show that for time in- dependent reaction eventsν(n, t) =ν(n), integration of the probabilityq?_{(n, t;dt)≡1−q(n, t;dt) gives}

q?(n, t;τ) = exp [−a(n)τ], as quoted in the main text (Eq. 4.29).

(b) Repeat part 6a fortime dependent reaction eventsν(n, t). (c) Write a stochastic simulation algorithm to generate realizations

of the stochastic process that describes the Brusselator in a growing cell. That is, repeat the example in the text, but with Ω a time-dependent quantity. In bacterial cell growth, the volume grows approximately exponentially over a cell-cycle, then divides more or less symmetrically.

i. As a first approximation, assume perfect division of the cell volume and perfect partitioning of the cell contents into daughter cells.

ii. Code a routine to allow both the volume after division and the partitioned contents to be narrowly-peaked random variables. What distribution will you choose for these two variables?

7. Exploring stochastic dynamics: Stochastic models exhibit fea- tures that do not appear in their deterministic counter-parts. Some of these features are straight-forward, others are quite surprising.

(a) Stable system. Consider a very simple system with constant synthesis and linear degradation,

x ν1

−→x+ 1, ν1=α

x ν2

−→x−1, ν2=β·x.

Starting withx(0) = 0, compute the deterministic trajectory for x(t). Generate stochastic simulation data for molecule

numbers from about 10−103_{, and plot these runs normalized}

to the same steady-state value. How does the relative magni- tude of the fluctuations scale with the number of molecules? Plot a histogram of the fluctuations around the steady-state – what distribution does it resemble? How does the half-width scale with the number of molecules?

(b) Multi-stable system. A “toggle switch” network consists of two mutually repressing species,r1 and r2 – Ifr1 is high, synthe-

sis of r2 is low, and, conversely, if r2 is high, r1 is kept low.

A simple network describing this system is the following four reactions: r1 ν1 −→r1+ 1, ν1=α·gR(r2/Ω) r2 ν2 −→r2+ 1, ν2=α·gR(r1/Ω) r1 ν3 −→r1−1, ν3=β· r1 Ω r2 ν4 −→r2−1, ν4=β· r2 Ω,

where the functiongR(x) is high ifxis low and low ifxis high.

SupposegR(x) takes the simple Hill-form,

gR(x) = 1 +f · x KR n 1 +_Kx R n ,

where f is the capacity, KR measures the repressor strength

(smaller the KR, the less repressor necessary to reduce gR),

and n is the cooperativity determining how abrupt the tran- sition is from the high to low state. Nondimensionalize the deterministic rate equations corresponding to this system, and estimate a range of parameters for which the system exhibits bistability. Perform stochastic simulations of the model for pa- rameters in the bistable regime, along with varying system size Ω. What differences do you see comparing the stochastic and deterministic models?

(c) Noise-induced oscillations. Read J. M. G. Vilar, H. Y. Kueh, N. Barkai, and S. Leibler (2002) Mechanisms of noise-resistance in genetic oscillators,Proceedings of the National Academy of Sciences USA99: 15988–15992. Repeat the simulation of their model for the parameter choice leading to a deterministically stable system.

CHAPTER

5

PERTURBATION EXPANSION OF THE

MASTER EQUATION

The master equation derived in Chapter 3 provides a foundation for most applications of stochastic processes to physical phenomena. Although it is more tractable than the Chapman-Kolmogorov equation, it is still rare to find an exact solution. One possibility is to adopt the Fokker-Planck equation as an approximate evolution equation, as in Chapter 6. We shall show in this Chapter that this is the first-step in a systematic analytic approximation scheme.

5.1

Linear Noise Approximation (LNA)

• N. G. van Kampen (1976) “Expansion of the Master equation,”Advances in

Chemical Physics34: 245.

Often we can gain a better sense of a particular model by examining certain limiting regimes. The approximation method that we describe in this section examines system behavior in the limit of large numbers of reactant molecules.

We have already seen that as the number of molecules increases, the system evolution becomes more smooth and the deterministic formulation becomes more appropriate (Figure 3.5). The linear noise approximation exploits this behavior and rests upon the supposition that the determin- istic evolution of the reactant concentrations, call them x, can be mean-

Ω → ∞ Large Volume

( )

x t

( )

α,t Π a) b)

Figure 5.1: The Linear Noise Approximation. a) The microscopic fluctuations are separated from the macroscopic evolution of the system by re-writing the probability density for the whole stateP(n, t) as a distri- bution for the fluctuations Π(α, t) centered on the macroscopic trajectory

x(t). b)The discrete state space is smeared into a continuum by replacing the discrete step-operatorEby a continuousdifferential operator. ingfully separated from the fluctuations, call themα, and that these fluc- tuations scale roughly as thesquare-root of the number of molecules. We introduce an extensive parameter Ω that carries the units of volume and is directly proportional to the molecule numbers, allowing the molecule numbers to be written

ni= Ωxi+

√

Ωαi. (5.1)

We are lead to the square-root scaling of the fluctuations by the sug-

In document Applied Stochastic Processes in Science and Engineering - Free Computer, Programming, Mathematics, Technical Books, Lecture Notes and Tutorials (Page 97-112)