We start with some definitions.
Definition A.1. Let I be either IIN or [0, ∞). A stochastic process on I with state space E is a family of random variables {Xt : t ∈ I} on E. Let E be a topological space. A stochastic process is called cadlag if it has a.s. right continuous paths and the limits from the left exist. It is called continuous if its paths are a.s. continuous. We will often identify a cadlag (continuous, respectively) stochastic process with a random variable on the space of cadlag (continuous) functions. For the rest of these notes we will always assume that all stochastic processes on [0, ∞) are cadlag.
Definition A.2. A cadlag stochastic process {Nt} on [0, ∞) is called point pro-cess if a.s.
i) N0 = 0,
ii) Nt ≥ Ns for all t ≥ s and iii) Nt ∈ IIN for all t ∈ (0, ∞).
We denote the jump times by T1, T2, . . ., i.e. Tk= inf{t ≥ 0 : Nt ≥ k} for all k ∈ IIN.
In particular T0 = 0. A point process is called simple if T0 < T1 < T2 < · · · . Definition A.3. A stochastic process {Xt} is said to have independent incre-ments if for all n ∈ IIN and all real numbers 0 = t0 < t1 < t2 < · · · < tn the random variables Xt1 − Xt0, Xt2 − Xt1, . . . , Xtn − Xtn−1 are independent. A stochastic pro-cess is said to have stationary increments if for all n ∈ IIN and all real numbers 0 = t0 < t1 < t2 < · · · < tn and all h > 0 the random vectors (Xt1 − Xt0, Xt2 − Xt1, . . . , Xtn− Xtn−1) and (Xt1+h− Xt0+h, Xt2+h− Xt1+h, . . . , Xtn+h− Xtn−1+h) have the same distribution.
Definition A.4. An increasing family {Ft} of σ-algebras is called filtration if Ft ⊂ F for all t. A stochastic process {Xt} is called {Ft}-adapted if Xt is Ft -measurable for all t ≥ 0. The natural filtration {FtX} of a stochastic process {Xt} is the smallest filtration such that the process is adapted. If a stochastic process is considered then we use its natural filtration if nothing else is mentioned. A filtration is called right continuous if Ft+ :=T
s>tFs = Ft for all t ≥ 0.
Definition A.5. A {Ft}-stopping time is a random variable T on [0, ∞] or IIN such that {T ≤ t} ∈ Ft for all t ≥ 0. If we stop a stochastic process then we usually assume that T is a stopping time with respect to the natural filtration of the stochastic process. The σ-algebra
FT := {A ∈ F : A ∩ {T ≤ t} ∈ Ft for all t ≥ 0}
is called the information up to the stopping time T .
A.1. Bibliographical Remarks
For further literature on stochastic processes see for instance [30], [39], [58], [59], [67] or [88].
B. Martingales
Definition B.1. A stochastic process {Mt} with state space IR is called a {Ft }-martingale (-sub}-martingale, -super}-martingale, respectively) if
i) {Mt} is adapted,
ii) IIE[Mt] exists for all t ≥ 0,
iii) IIE[Mt | Fs] = (≥, ≤) Ms a.s. for all t ≥ s ≥ 0.
We simply say {Mt} is a martingale (submartingale, supermartingale) if it is a martingale (submartingale, supermartingale) with respect to its natural filtration.
The following two propositions are very important for dealing with martingales.
Proposition B.2. (Martingale stopping theorem) Let {Mt} be a {Ft }-martingale (-sub}-martingale, -super}-martingale, respectively) and T be a {Ft }-stop-ping time. Assume that {Ft} is right continuous. Then also the stochastic process {MT ∧t : t ≥ 0} is a {Ft}-martingale (-submartingale, -supermartingale). Moreover, IIE[Mt| FT] = (≥, ≤)MT ∧t.
Proof. See [30, Thm. 12], [39, Thm. II.2.13] or [66, Thm. 10.2.5].
Proposition B.3. (Martingale convergence theorem) Let {Mt} be a {Ft }-martingale such that lim
t→∞IIE[Mt−] < ∞ (or equivalently sup
t≥0
IIE[|Mt|] < ∞). If {Ft} is right continuous then the random variable M∞ := lim
t→∞Mt exists a.s. and is inte-grable.
Proof. See for instance [30, Thm. 6] or [66, Thm. 10.2.2].
Note that in general
IIE[M∞] = IIE[ lim
t→∞Mt] 6= lim
t→∞IIE[Mt] = IIE[M0] .
B.1. Bibliographical Remarks
The theory of martingales can also be found in [30], [39] or [66].
C. Renewal Processes C.1. Poisson Processes
Definition C.1. A point process {Nt} is called (homogeneous) Poisson process with rate λ if
i) {Nt} has stationary and independent increments,
ii) IIP[Nh = 0] = 1 − λh + o(h) as h → 0,
iii) IIP[Nh = 1] = λh + o(h) as h → 0.
Remark. It follows readily that IIP[Nh ≥ 2] = o(h) and that the point process is
simple.
We give now some alternative definitions of the Poisson process.
Proposition C.2. Let {Nt} be a point process. Then the following are equivalent:
i) {Nt} is a Poisson process with rate λ.
ii) {Nt} has independent increments and Nt∼ Pois(λt) for each fixed t ≥ 0.
iii) The interarrival times {Tk − Tk−1 : k ≥ 1} are independent and Exp(λ) dis-tributed.
iv) For each fixed t ≥ 0, Nt ∼ Pois(λt) and given {Nt = n} the occurrence points have the same distribution as the order statistics of n independent uniformly on [0, t] distributed points.
v) {Nt} has independent increments such that IIE[N1] = λ and given {Nt = n} the occurrence points have the same distribution as the order statistics of n indepen-dent uniformly on [0, t] distributed points.
vi) {Nt} has independent and stationary increments such that IIP[Nh ≥ 2] = o(h) and IIE[N1] = λ.
Proof. “i) =⇒ ii)” Let pn(t) = IIP[Nt= n]. Then show that pn(t) is continuous and differentiable. Finding the differential equations and solving them shows ii).
The details are left as an exercise.
“ii) =⇒ iii)” We first show that {Nt} has stationary increments. It is enough to show that Nt+h− Nh is Pois(λt) distributed. For the moment generating functions we obtain
eλ(t+h)(er−1) = IIE[er(Nt+h−Nh)erNh] = IIE[er(Nt+h−Nh)]eλh(er−1). It follows that
IIE[er(Nt+h−Nh)] = eλt(er−1). This proves that Nt+h− Nh is Pois(λt) distributed.
Let t0 = 0 ≤ s1 < t1 ≤ s2 < t2 ≤ · · · ≤ sn < tn. Then
and therefore they are independent and Exp(λ) distributed.
“iii) =⇒ iv)” Note that Tn is Γ(n, λ) distributed. Thus IIP[Nt= 0] = IIP[T1 > t] =
The joint density of T1, . . . , Tn+1 is for t0 = 0 This is the claimed conditional distribution.
“iv) =⇒ v)” It is clear that IIE[N1] = λ. Let xk ∈ IIN and t0 = 0 < t1 < · · · < tn. and therefore {Nt} has independent increments.
“v) =⇒ i)” Note that for xk ∈ IIN, t0 = 0 < t1 < · · · < tn and h > 0 IIP[Ntk+h− Ntk−1+h= xk, 1 ≤ k ≤ n | Ntn+h]
= IIP[Ntk − Ntk−1 = xk, 1 ≤ k ≤ n | Ntn+h] .
Integrating with respect to Ntn+h yields that {Nt} has stationary increments. For IIP[Nt= 0] we obtain Reordering the terms yields
e−λ0(t−h)− e−λ0t
that λ0 = IIE[N1] = λ. This implies i).
“i) =⇒ vi)” Follows readily from i) and v).
“vi) =⇒ i)” As in the proof of “v) =⇒ i)” it follows that IIP[Nt= 0] = e−λ0t. Then IIP[Nh = 1] = 1 − IIP[Nh = 0] − IIP[Nh ≥ 2] = λ0h + o(h). Thus {Nt} is a Poisson process with rate λ0. From “i) ⇐⇒ v)” we conclude that λ0 = IIE[N1] = λ.
Remark. The condition IIE[N1] = λ in v) and vi) is only used for identifying the parameter λ. We did not use in the proof that IIE[N1] < ∞.
The Poisson process has the following properties.
Proposition C.3. Let {Nt} and { ˜Nt} be two independent Poisson processes with rates λ and ˜λ respectively. Let {Ii : i ∈ IIN} be an iid. sequence of random variables independent of {Nt} with IIP[Ii = 1] = 1 − IIP[Ii = 0] = q for some q ∈ (0, 1).
Furthermore let a > 0 be a real number. Then i) {Nt+ ˜Nt} is a Poisson process with rate λ + ˜λ.
ii) nXNt
i=1
Iio
is a Poisson process with rate λq.
iii) {Nat} is a Poisson process with rate λa.
Proof. Exercise.
Definition C.4. Let Λ(t) be an increasing right continuous function on [0, ∞) with Λ(0) = 0. A point process {Nt} on [0, ∞) is called inhomogeneous Poisson process with intensity measure Λ(t) if
i) {Nt} has independent increments, ii) Nt− Ns∼ Pois(Λ(t) − Λ(s)).
If there exists a function λ(t) such that Λ(t) =Rt
0 λ(s) ds then λ(t) is called inten-sity or rate of the inhomogeneous Poisson process.
Note that a homogeneous Poisson process is a special case with Λ(t) = λt. Define Λ−1(x) = sup{t ≥ 0 : Λ(t) ≤ x} the inverse function of Λ(t).
Proposition C.5. Let { ˜Nt} be a homogeneous Poisson process with rate 1. Define Nt = ˜NΛ(t). Then {Nt} is an inhomogeneous Poisson process with intensity mea-sure Λ(t). Conversely, let {Nt} be an inhomogeneous Poisson process with intensity measure Λ(t). Let ˜Nt = NΛ−1(t) at all points where Λ(Λ−1(t)) = t. On inter-vals (Λ(Λ−1(t)−), Λ(Λ−1(t))) where Λ(Λ−1(t)) 6= t let there be NΛ−1(t) − N(Λ−1(t)−) occurrence points uniformly distributed on the interval (Λ(Λ−1(t)−), Λ(Λ−1(t))) in-dependent of {Nt}. Then { ˜Nt} is a homogeneous Poisson process with rate 1.
Proof. Exercise.
For an inhomogeneous Poisson process we can construct the following martin-gales.
Lemma C.6. Let r ∈ IR. The following processes are martingales.
i) {Nt− Λ(t)},
ii) {(Nt− Λ(t))2− Λ(t)}, iii) {exp{rNt− Λ(t)(er− 1)}}.
Proof. i) Because {Nt} has independent increments
IIE[Nt− Λ(t) | Fs] = IIE[Nt− Ns] + Ns− Λ(t) = Ns− Λ(s) . ii) Analogously,
IIE[(Nt− Λ(t))2− Λ(t) | Fs]
= IIE[(Nt− Ns− {Λ(t) − Λ(s)} + Ns− Λ(s))2 | Fs] − Λ(t)
= IIE[(Nt− Ns− {Λ(t) − Λ(s)})2]
+ 2(Ns− Λ(s))IIE[Nt− Ns− {Λ(t) − Λ(s)}] + (Ns− Λ(s))2− Λ(t)
= Λ(t) − Λ(s) + 2 · 0 + (Ns− Λ(s))2 − Λ(t) = (Ns− Λ(s))2− Λ(s) . iii) Analogously
IIE[erNt−Λ(t)(er−1) | Fs] = IIE[er(Nt−Ns)]erNs−Λ(t)(er−1) = erNs−Λ(s)(er−1).
C.2. Renewal Processes
Definition C.7. A simple point process {Nt} is called an ordinary renewal process if the interarrival times {Tk− Tk−1 : k ≥ 1} are iid.. If T1 has a different distribution then {Nt} is called a delayed renewal process. If λ−1 = IIE[T2− T1] exists and
IIP[T1 ≤ x] = λ Z x
0
IIP[T2− T1 > y] dy (C.1) then {Nt} is called a stationary renewal process. If {Nt} is an ordinary renewal process then the function U (t) = 1I{t≥0}+ IIE[Nt] is called the renewal measure.
In the rest of this section we denote by F the distribution function of T2 − T1. For simplicity we let T be a random variable with distribution F . Note that because the point process is simple we implicitly assume that F (0) = 0. If nothing else is said we consider in the sequel only ordinary renewal processes.
Recall that F∗0 is the indicator function of the interval [0, ∞).
Lemma C.8. The renewal measure can be written as U (t) =
∞
X
n=0
F∗n(t) .
Moreover, U (t) < ∞ for all t ≥ 0 and U (t) → ∞ as t → ∞.
Proof. Exercise.
In the renewal theory one often has to solve equations of the form Z(x) = z(x) +
Z x 0
Z(x − y) dF (y) (x ≥ 0) (C.2) where z(x) is a known function and Z(x) is unknown. This equation is called the renewal equation. The equation can be solved explicitly.
Proposition C.9. If z(x) is bounded on bounded intervals then Z(x) =
Z x 0−
z(x − y) dU (y) = z ∗ U (x)
is the unique solution to (C.2) that is bounded on bounded intervals.
Proof. We first show that Z(x) is a solution. This follows from Let Z1(x) be a solution that is bounded on bounded intervals. Then
|Z1(x)−Z(x)| =
Let z(x) be a bounded function and h > 0 be a real number. Define
mk(h) = sup{z(t) : (k − 1)h ≤ t < kh}, mk(h) = inf{z(t) : (k − 1)h ≤ t < kh}
and the Riemann sums σ (h) = h
Definition C.10. A function z(x) is called directly Riemann integrable if
−∞ < lim
h↓0
σ(h) = lim
h↓0σ (h) < ∞ .
The following lemma gives some criteria for a function to be directly Riemann inte-grable.
Lemma C.11.
• The space of directly Riemann integrable functions is a linear space.
• If z(t) is monotone and R∞
0 z(t) dt < ∞ then z(t) is directly Riemann integrable.
• If a(t) and b(t) are directly Riemann integrable and z(t) is continuous Lebesgue a.e. such that a(t) ≤ z(t) ≤ b(t) then z(t) is directly Riemann integrable.
• If z(t) ≥ 0, z(t) is continuous Lebesgue a.e. and σ (h) < ∞ for some h > 0 then is z(t) directly Riemann integrable.
Proof. See for instance [1, p.69].
Definition C.12. A distribution function F of a positive random variable X is called arithmetic if for some γ one has IIP[X ∈ {γ, 2γ, . . .}] = 1. The span γ is the largest number such that the above relation is fulfilled.
The most important result in renewal theory is a result on the asymptotic behaviour of the solution to (C.2).
Proposition C.13. (Renewal theorem) If z(x) is a directly Riemann inte-grable function then the solution Z(x) to the renewal equation (C.2) satisfies
t→∞lim Z(t) = λ Z ∞
0
z(y) dy if F is non-arithmetic and
n→∞lim Z(t + nγ) = γλ
Proof. See for instance [40, p.364] or [66, p.218].
Example C.14. Assume that F is non-arithmetic and that IIE[T12] < ∞. We consider the function Z(t) = IIE[TNt+1− t], the expected time till the next occurrence of an event. Consider first IIE[TNt+1− t | T1 = s]. If s ≤ t then there is a renewal The function is monotone and
Z ∞ Thus z(t) is directly Riemann integrable and by the renewal theorem
t→∞lim Z(t) = λ
2IIET12 .
Example C.15. Assume that λ > 0 and that F is non-arithmetic. What is the asymptotic distribution of the waiting time till the next event? Let x > 0 and set Z(t) = IIP[TNt+1− t > x]. Conditioning on T1 = s yields
IIP[TNt+1− t > x | T1 = s] =
Z(t − s) if s ≤ t,
0 if t < s ≤ t + x, 1 if t + x < s.
This gives the renewal equation
Z(t) = 1 − F (t + x) + Z t
0
Z(t − s) dF (s) .
z(t) = 1 − F (t + x) is decreasing in t and its integral is bounded by λ−1. Thus it is directly Riemann integrable. It follows from the renewal theorem that
t→∞lim Z(t) = λ Z ∞
0
(1 − F (t + x)) dt = λ Z ∞
x
(1 − F (t)) dt .
Thus the stationary distribution of the next event must be the distribution given by
(C.1).
C.3. Bibliographical Remarks
The theory of point processes can also be found in [19] or [26]. For further literature on renewal theory see also [1] and [40].
D. Brownian Motion
Definition D.1. A stochastic process {Wt} is called a (m, η2)-Brownian motion if a.s.
• W0 = 0,
• {Wt} has independent increments,
• Wt∼ N(mt, η2t) and
• {Wt} has cadlag paths.
A (0, 1)-Brownian motion is called a standard Brownian motion.
It can be shown that Brownian motion exists. Moreover, one can proof that a Brownian motion has continuous paths.
From the definition it follows that {Wt} has also stationary increments.
Lemma D.2. A (m, η2)-Brownian motion has stationary increments.
Proof. Exercise.
We construct the following martingales.
Lemma D.3. Let {Wt} be a (m, η2)-Brownian motion and r ∈ IR. The following processes are martingales.
i) {Wt− mt}.
ii) {(Wt− mt)2− η2t}.
iii) {exp{r(Wt− mt) − η22r2t}}.
Proof. i) Using the stationary and independent increments property IIE[Wt− mt | Fs] = IIE[Wt− Ws] + Ws− mt = Ws− ms . ii) It suffices to consider the case m = 0.
IIE[Wt2− η2t | Fs] = IIE[(Wt− Ws)2+ 2(Wt− Ws)Ws+ Ws2 | Fs] − η2t = Ws2− η2s . iii) It suffices to consider the case m = 0.
IIE[erWt−η22 r2t | Fs] = IIE[er(Wt−Ws)]erWs−η22 r2t= erWs−η22r2s.