Stochastic Processes
6.3 Counting processes
A large number of situations can be described by counting processes. As becomes clear from their name, they count. What they count differs greatly from one process to another. These may be counts of arrived jobs, completed tasks, transmitted messages, detected errors, scored goals, and so on.
DEFINITION6.10
A stochastic process X is counting if X(t) is the number of items counted by the time t.
As time passes, one can count additional items; therefore, sample paths of a counting process are always non-decreasing. Also, counts are nonnegative integers, X(t) ∈ {0, 1, 2, 3, ...}.
Hence, all counting processes are discrete-state.
Example 6.17 (E-mails and attachments). Figure 6.6 shows sample paths of two counting process, X(t) being the number of transmitted e-mails by the time t and A(t) being the number of transmitted attachments. According to the graphs, e-mails were trans-mitted at t = 8, 22, 30, 32, 35, 40, 41, 50, 52, and 57 min. The e-mail counting process X(t) increments by 1 at each of these times. Only 3 of these e-mails contained attachments. One attachment was sent at t = 8, five more at t = 35, making the total of A(35) = 6, and two more attachments at t = 50, making the total of A(50) = 8. ♦
Two classes of counting processes will be discussed in detail, a discrete-time Binomial process and a continuous-time Poisson process.
6.3.1 Binomial process
We again consider a sequence of independent Bernoulli trials with probability of success p (Sections 3.4.1–3.4.4) and start counting “successes.”
✲
✻
0 2 4 6 8 10
0 10 20 30 40 50 t (min) X(t)
✲
✻
0 2 4 6 8 10
0 10 20 30 40 50 t (min) A(t)
FIGURE 6.6: Counting processes in Example 6.17.
✲
FIGURE 6.7: Binomial process (sample path). Legend:S= success,F=failure.
DEFINITION6.11
Binomial process X(n) is the number of successes in the first n independent Bernoulli trials, where n = 0, 1, 2, . . ..
It is a discrete-time discrete-space counting stochastic process. Moreover, it is Markov, and therefore, a Markov chain.
As we know from Sections 3.4.2–3.4.3, the distribution of X(n) at any time n is Binomial(n, p), and the number of trials Y between two consecutive successes is Geometric(p) (Figure 6.7).
Notation X(n) = number of successes in n trials
Y = number of trials between consecutive successes
Relation to real time: frames
The “time” variable n actually measures the number of trials. It is not expressed in minutes or seconds. However, it can be related to real time.
Suppose that Bernoulli trials occur at equal time intervals, every ∆ seconds. Then n trials occur during time t = n∆, and thus, the value of the process at time t has Binomial distribution with parameters n = t/∆ and p. The expected number of successes during t seconds is therefore
Arrival rate λ = p/∆ is the average number of successes per one unit of time.
The time interval ∆ of each Bernoulli trial is called a frame. The interarrival time is the time between successes.
These concepts, arrival rate and interarrival time, deal with modeling arrivals of jobs, mes-sages, customers, and so on with a Binomial counting process, which is a common method in discrete-time queuing systems (Section 7.3). The key assumption in such models is that no more than 1 arrival is allowed during each ∆-second frame, so each frame is a Bernoulli trial. If this assumption appears unreasonable, and two or more arrivals can occur during the same frame, one has to model the process with a smaller ∆.
Notation λ = arrival rate
∆ = frame size
p = probability of arrival (success) during one frame (trial) X(t/∆) = number of arrivals by the time t
T = interarrival time
The interarrival period consists of a Geometric number of frames Y , each frame taking ∆ seconds. Hence, the interarrival time can be computed as
T = Y ∆.
It is a rescaled Geometric random variable taking possible values ∆, 2∆, 3∆, etc. Its ex-pectation and variance are
E(T ) = E(Y )∆ = 1 p∆ = 1
λ; Var(T ) = Var(Y )∆2= (1− p)
∆ p
2
or 1− p λ2 .
Binomial counting process
λ = p/∆
n = t/∆
X(n) = Binomial(n, p) Y = Geometric(p)
T = Y ∆
Example 6.18 (Mainframe computer). Jobs are sent to a mainframe computer at a rate of 2 jobs per minute. Arrivals are modeled by a Binomial counting process.
(a) Choose such a frame size that makes the probability of a new job during each frame equal 0.1.
(b) Using the chosen frames, compute the probability of more than 3 jobs received during one minute.
(c) Compute the probability of more than 30 jobs during 10 minutes.
(d) What is the average interarrival time, and what is the variance?
(e) Compute the probability that the next job does not arrive during the next 30 seconds.
Solution.
(a) We have λ = 2 min−1 and p = 0.1. Then
∆ = p
λ = 0.05 min or 3 sec.
(b) During t = 1 min, we have n = t/∆ = 20 frames. The number of jobs during this time is Binomial(n = 20, p = 0.1). From Table A2 (in the Appendix),
P{X(n) > 3} = 1 − P {X(n) ≤ 3} = 1 − 0.8670 = 0.1330.
(c) Here n = 10/0.05 = 200 frames, and we use Normal approximation to Binomial(n, p) distribution (recall p. 94). With a proper continuity correction, and using Table A4,
P{X(n) > 30} = P {X(n) > 30.5}
= P
( X(n)− np
pnp(1− p) > 30.5− (200)(0.1) p(200)(0.1)(1− 0.1)
)
= P{Z > 2.48} = 1 − 0.9934 = 0.0066.
Comparing questions (b) and (c), notice that 3 jobs during 1 minute is not the same as 30 jobs during 10 minutes!
(d) E(T ) = 1/λ = 1/2 min or 30 sec. Intuitively, this is rather clear because the jobs arrive at a rate of two per minute. The interarrival time has variance
Var(T ) = 1− p λ2 = 0.9
22 = 0.225.
(e) For the interarrival time T = Y ∆ = Y (0.05) and a Geometric variable Y , P{T > 0.5 min} = P {Y (0.05) > 0.5} = P {Y > 10}
= X∞ k=11
(1− p)k−1p = (1− p)10= (0.9)10= 0.3138.
Alternatively, this is also the probability of 0 arrivals during n = t/∆ = 0.5/0.05 = 10
frames, which is also (1− p)10. ♦
Markov property
Binomial counting process is Markov, with transition probabilities
pij=
p if j = i + 1 1− p if j = i
0 otherwise
That is, during each frame, the number of successes X increments by 1 in case of a success or remains the same in case of a failure (see Figure 6.8). Transition probabilities are constant over time and independent of the past values of X(n); therefore, it is a stationary Markov chain.
0 ✲ 1 ✲ 2 ✲ 3 ✲
❄ ❄ ❄ ❄
p p p p
1− p 1− p 1− p 1− p
FIGURE 6.8: Transition diagram for a Binomial counting process.
This Markov chain is irregular because X(n) is non-decreasing, thus p(h)10 = 0 for all h. Once we see one success, the number of successes will never return to zero. Thus, this process has no steady-state distribution.
The h-step transition probabilities simply form a Binomial distribution. Indeed, p(h)ij is the probability of going from i to j successes in h transitions, i.e.,
p(h)ij = P{(j − i) successes in h trials}
Notice that the transition probability matrix has ∞ rows and ∞ columns because X(n) can reach any large value for sufficiently large n:
P =
We now turn attention to continuous-time stochastic processes. The time variable t will run continuously through the whole interval, and thus, even during one minute there will be infinitely many moments when the process X(t) may change. Then, how can one study such models?
Often a continuous-time process can be viewed as a limit of some discrete-time process whose frame size gradually decreases to zero, therefore allowing more frames during any fixed period of time,
∆↓ 0 and n ↑ ∞.
Example 6.19. For illustration, let us recall how movies are made. Although all motions on the screen seem continuous, we realize that an infinite amount of information could not be stored in a video cassette. Instead, we see a discrete sequence of exposures that run so fast that each motion seems continuous and smooth.
∆ = 0.16 sec ∆ = 0.04 sec FIGURE 6.9: From discrete motion to continuous motion: reducing the frame size ∆.
Early-age video cameras shot exposures rather slowly; the interval ∆ between successive shots was pretty long (∼ 0.2–0.5 sec). As a result, the quality of recorded video was rather low. Movies were “too discrete.”
Modern camcorders can shoot up to 200 exposures per second attaining ∆ = 0.005. With such a small ∆, the resulting movie seems perfectly continuous. A shorter frame ∆ results
in a “more continuous” process (Figure 6.9). ♦
Poisson process as the limiting case
Going from discrete time to continuous time, Poisson process is the limiting case of a Binomial counting process as ∆↓ 0.
DEFINITION6.13
Poisson process is a continuous-time counting stochastic process obtained from a Binomial counting process when its frame size ∆ decreases to 0 while the arrival rate λ remains constant.
We can now study a Poisson process by taking a Binomial counting process and letting its frame size decrease to zero.
Consider a Binomial counting process that counts arrivals or other events occurring at a rate λ. Let X(t) denote the number of arrivals occurring until time t. What will happen with this process if we let its frame size ∆ converge to 0?
The arrival rate λ remains constant. Arrivals occur at the same rate (say, messages arrive at a server) regardless of your choice of frame ∆.
The number of frames during time t increases to infinity, n = t
∆ ↑ ∞ as ∆ ↓ 0.
The probability of an arrival during each frame is proportional to ∆, so it also decreases to 0,
p = λ∆↓ 0 as ∆ ↓ 0.
Then, the number of arrivals during time t is a Binomial(n, p) variable with expectation E X(t) = np =tp
∆ = λt.
In the limiting case, as ∆ ↓ 0, n ↑ ∞, and p ↓ 0, it becomes a Poisson variable with parameter np = λt,
X(t) = Binomial(n, p)→ Poisson(λ) (if in doubt, see p. 66).
The interarrival time T becomes a random variable with the c.d.f.
FT(t) = P{T ≤ t} = P {Y ≤ n} because T = Y ∆ and t = n∆
= 1− (1 − p)n Geometric distribution of Y
= 1−
1−λt
n
n
because p = λ∆ = λt/n
→ 1 − eλt. This is the “Euler limit”:
(1 + x/n)n → ex as n→ ∞
What we got is the c.d.f. of Exponential distribution! Hence, the interarrival time is Expo-nential with parameter λ.
Further, the time Tkof the k-th arrival is the sum of k Exponential interrarival times that has Gamma(k, λ) distribution. From this, we immediately obtain our familiar Gamma-Poisson formula (4.14),
P{Tk ≤ t} = P { k-th arrival before time t } = P {X(t) ≥ k}
where Tk is Gamma(k, λ) and X(t) is Poisson(λt).
Poisson process
X(t) = Poisson(λt) T = Exponential(λ) Tk = Gamma(k, λ) P{Tk ≤ t} = P {X(t) ≥ k}
P{Tk > t} = P {X(t) < k}
A sample path of some Poisson process is shown in Figure 6.10.
Example 6.20 (Web site hits). The number of hits to a certain web site follows a Poisson process with the intensity parameter λ = 7 hits per minute.
On the average, how much time is needed to get 10,000 hits? What is the probability that this will happen within 24 hours?
Solution. The time of the 10, 000-th hit Tk has Gamma distribution with parameters k = 10, 000 and λ = 7 min−1. Then, the expected time of the k-th hit is
µ = E(Tk) = k
λ = 1,428.6 min or 23.81 hrs.
✲
FIGURE 6.10: Poisson process (sample path).
Also,
σ = Std(Tk) =
√k
λ = 14.3 min.
By the Cental Limit Theorem of Section 4.3, we can use the Normal approximation to the Gamma distribution of Tk. The probability that the 10,000-th hit occurs within 24 hours (1440 min) is
A more conventional definition of a Poisson process sounds like this.
DEFINITION6.14
Poisson process X(t) is a continuous-time counting process with independent increments, such that
(a) P{X(t + ∆) − X(t) = 1} = λ∆ + o(∆) as ∆ → 0;
(b) P{X(t + ∆) − X(t) > 1} = o(∆) as ∆ → 0.
In this definition, differences X(t + ∆)− X(t) are called increments. For a Poisson process, an increment is the number of arrivals during time interval (t, t + ∆].
Properties (a) and (b) imply the known fact that Binomial process probabilities differ from their limits (as ∆→ 0) by a small quantity converging to 0 faster than ∆. This quantity is in general denoted by o(∆);
o(∆)
∆ → 0 as ∆ → 0.
For a Binomial counting process, P{X(t + ∆) − X(t) = 1} is the probability of 1 arrival
during 1 frame, and it equals p = λ∆, whereas the probability of more than 1 arrival is zero. For the Poisson process, these probabilities may be different, but only by a little.
The last definition describes formally what we called rare events. These events occur at random times; probability of a new event during a short interval of time is proportional to the length of this interval. Probability of more than 1 event during that time is much smaller comparing with the length of such interval. For such sequences of events a Poisson process is a suitable stochastic model. Examples of rare events include telephone calls, message arrivals, virus attacks, errors in codes, traffic accidents, natural disasters, network blackouts, and so on.
Example 6.21 (Mainframe computer, revisited). Let us now model the job arrivals in Example 6.18 on p. 150 with a Poisson process. Keeping the same arrival rate λ = 2 min−1, this will remove a perhaps unnecessary assumption that no more than 1 job can arrive during any given frame.
Revisiting questions (b)–(e) in Example 6.18, we now obtain, (b) The probability of more than 3 jobs during 1 minute is
P{X(1) > 3} = 1 − P {X(1) ≤ 3} = 1 − 0.8571 = 0.1429, from Table A3 with parameter λt = (2)(1) = 2.
(c) The probability of more than 30 jobs during 10 minutes is
P{X(10) > 30} = 1 − P {X(10) ≤ 30} = 1 − 0.9865 = 0.0135, from Table A3 with parameter λt = (2)(10) = 20.
(d) For the Exponential(λ = 2) distribution of interarrival times, we get again that E(T ) = 1
λ = 0.5 min or 30 sec.
This is because the jobs arrive to the computer at the same rate regardless of whether we model their arrivals with a Binomial or Poisson process. Also,
Var(T ) = 1
λ2 = 0.25.
(e) The probability that the next job does not arrive during the next 30 seconds is P{T > 0.5 min} = e−λ(0.5) = e−1 = 0.3679.
Alternatively, this is also the probability of 0 arrivals during 0.5 min, i.e., P{X(0.5) = 0} = 0.3679,
from Table A3 with parameter λt = (2)(0.5) = 1.
We see that most of the results are somewhat different from Example 6.18, where the same events were modeled by a Binomial counting process. Noticeably, the variance of interarrival times increased. Binomial process introduces a restriction on the number of arrivals during
each frame, therefore reducing variability. ♦