Sample Paths, Convergence, and Averages
Definition 5.10 An ergodic system is one that is positive recurrent, aperiodic, and
5.3 time average versus ensemble average 89 These terms are all defined precisely in Chapter9, but we explain the ideas now. We start with irreducibility. Irreducibility says that a process should be able to get from any state to any other state (think of the state as the number of jobs in the system). This is important for ensuring that the choice of initial state does not matter.
The positive recurrent property is the most important condition with respect to under- standing the equivalence between the time average and the ensemble average: Given an irreducible system, in which we can get to any state, the system is positive recurrent if for any statei, the state is revisited infinitely often, and the mean time between visits to statei(renewals) is finite. Furthermore, every time that we visit stateithe system will probabilistically restart itself.
Question: Give an example of what it means for the process to probabilistically restart
itself.
Answer: Every time that the system empties (0jobs), the process starts anew in state
0. We call this a “restart.” In a positive recurrent system, the system empties infinitely often. This includes all systems that we will study.
Consider our example of a queue, where a new job is created at each time step with probabilitypand a job is removed with probabilityq > p. We start out with zero jobs in the system. We now start flipping coins, and the number of jobs in the system goes up and down. At some point, the number of jobs in the system returns to zero. At this point, we can imagine the same statistical process starting over again. And the next time the number of jobs in the system returns to zero, it will start over again. Thus a single long run of the system (Tim’s view) actually appears to be an infinite number of statistically independent runs (Enzo’s view).
The aperiodicity condition is important in making sure that the ensemble average exists. Aperiodicity refers to the fact that the system state (number of jobs in the system) should not be tied in some particular way to the time step; for example, it should not be the case that the system is always in state0for even time steps and state1
for odd time steps; otherwise, the particulartthat Enzo picked for stopping the system might sway his result.
Question: Explain intuitively why an ergodic system should have the property that the
time average equals the ensemble average.
Answer: Consider the time average over a single long run of the system as shown in
Figure5.7. This run can be thought of as a chain of many independent but statistically identical runs, each called a “renewal.” Let X1 represent the time average over just
Restart Restart Restart Restart Restart
90 sample paths, convergence, and averages
the first renewal, letX2 represent the time average over just the second renewal, etc.
Then the overall time average is the average ofX1, X2, . . . .But these are i.i.d. So,
by the Strong Law of Large Numbers (SLLN), the average of these converges with probability 1 to the expected time average over a single renewal, where the expectation is an ensemble average (taken over all sample paths).
5.3.5 Simulation
What does all this say about how we do simulation? Tim’s method of sampling a single process over a very long period of time and averaging those samples results in
NTime Avg. Enzo’s method of generating many independent processes and taking their average at some far-out timetyieldsNEnsemble. If these yield the same result, should we go for the easier method?
Question: The ensemble average seems more costly to compute, because we need new
random seeds. Why bother with the ensemble average if it comes out to the same thing as the time average?
Answer: The main reason is that the ensemble average can be obtained in parallel,
by running simulations on different cores or different machines. Another reason for using the ensemble average is that the independent data points allow us to generate confidence intervals, which allow us to bound the deviation in our result.
Question: Why in both the definitions of ensemble average and time average is it so
important that the system be run for a “long” time?
Answer: We want to get to the point where the initial state has no effect: We want to
reach “steady state.” This will all become more clear when we get to Markov chain theory.
5.3.6 Average Time in System
So far, we have talked about the average number of jobs in the system. We can also define two versions of the average time in system as follows:
TTime Avg= lim t→∞
A (t) i= 1 Ti
A(t) ,
whereTiis the time in system of theith arrival andA(t)is the number of arrivals by timet. Again the time average is assumed to be associated with a single sample path.
TEnsemble= lim
i→∞E [Ti] ,
whereE [Ti]is the average time in system of theith job, where the average is taken over all sample paths.
5.5 exercise 91
5.4 Related Readings
The following books provide more detail on the information covered in this chapter: Karlin and Taylor (pp. 474–89) [105], and Gross and Harris (pp. 38–45) [75].
5.5 Exercise
5.1 Weak Law of Large Numbers
LetX1, X2, X3, . . . , be i.i.d. random variables with finite meanE [X] and
finite varianceσ2. Your goal is to prove the Weak Law of Large Numbers:
∀, lim n→∞P ## ##Sn n − E [X] ## ## > = 0 whereSn = n i= 1Xi.
(a) Start out by proving Markov’s Inequality, which says: IfXis non-negative then
P {X > t} ≤ E [X]
t , ∀t > 0.
(b) Now use Markov’s Inequality to prove Chebyshev’s Inequality, which says: LetY be a random variable with finite meanE [Y ]and finite varianceσ2
Y. Then P {|Y − E [Y ] | ≥ t} ≤ σ 2 Y t2 .
(c) Finally use Chebyshev’s Inequality to prove the Weak Law of Large Numbers.
PART III