1.6 Stochastic Processes
1.6.6 Martingales
Martingales are an important class of stochastic processes. The concept of conditional expectation is important in developing a theory of martingales. Martingales are special sequences of random variables that have applications in various processes that evolve over time.
Definition 1.63 (martingale, submartingale, supermartingale) Let {Ft} be a filtration and let {Xt} be adapted to the filtration{Ft}. We say the sequence {(Xt,Ft) : t∈ T } is amartingaleiff
E(Xt|Ft−1) a.s.
= Xt−1. (1.276)
We say the sequence {(Xt,Ft) : t∈ T }is asubmartingale iff E(Xt|Ft−1)
a.s.
≥ Xt−1. (1.277)
We say the sequence {(Xt,Ft) : t∈ T }is asupermartingaleiff E(Xt|Ft−1)
a.s.
We also refer to a sequence of random variables{Xt:t∈ T }as a (sub, su- per)martingale if{(Xt, σ({Xs:s≤t})) :t∈ T }is a (sub, super)martingale; that is, the martingale is the sequence{Xt} instead of{(Xt,Ft)}, and a cor- responding sequence {Ft}is implicitly defined as{σ({Xs:s≤t})}.
This is consistent with the definition of {(Xt,Ft) : t ∈ T } as a (sub, super)martingale because clearly
σ({Xs : s≤r})r⊆σ({Xs : s≤t})t ifr≤t
(and so{σ({Xs : s≤t})t} is a filtration), and furthermore{Xt}is adapted to the filtration{σ({Xs : s≤t})t}.
We often refer to the type of (sub, super)martingale defined above as a
forward (sub, super)martingale. We define a reverse martingale analogously with the conditionsFt⊃ Ft+1⊃ · · · and E(Xt−1|Ft)
a.s. = Xt.
The sequence of sub-σ-fields, which is a filtration, is integral to the defini- tion of martingales. Given a sequence of random variables{Xt}, we may be interested in another sequence of random variables {Yt} that are related to theXs. We say that {Yt}is a martingale with respect to {Xt}if
E(Yt|{Xτ : τ≤s})a.s.= Ys, ∀s≤t. (1.279) We also sometimes define martingales in terms of a more general sequence of σ-fields. We may say that {Xt : t ∈ T } is a martingale relative to the sequence ofσ-fields{Dt : t∈ T } in some probability space (Ω,F, P), if
Xs= E(Xt|Dt) fors > t. (1.280) Submartingales and supermartingales relative to{Dt : t∈ T }may be defined analogously.
Example 1.33 Polya’s urn process
Consider an urn that initially contains r red and b blue balls, and Polya’s urn process (Example 1.6 on page 24). In this process, one ball is chosen randomly from the urn, and its color noted. The ball is then put back into the urn together withcballs of the same color. LetXn be the number of red balls in the urn afterniterations of this procedure, and letYn =Xn/(nc+r+b). Then the sequence {Yn}is a martingale (Exercise 1.82).
Interestingly, ifc >0, then {Yn} converges to the beta distribution with parametersr/candb/c; seeFreedman(1965). Freedman also discusses a vari- ation on Polya’s urn process called Friedman’s urn process, which is the same as Polya’s, except that at each draw in addition to the c balls of the same color being added to the urn, d balls of the opposite color are added to the urn. Remarkably, the behavior is radically different, and, in fact, ifc >0 and d >0, thenYn
a.s. →1/2.
Example 1.34 likelihood ratios
Let f and g be probability densities. Let X1, X2, . . .be an iid sequence of random variables whose range is within the intersection of the domains off andg. Let Yn= n Y i=1 g(Xi)/f(Xi). (1.281)
(This is called a “likelihood ratio” and has applications in statistics. Note that f(x) andg(x) are likelihoods, as defined in equation (1.19) on page20, although the “parameters” are the functions themselves.) Now suppose that f is the PDF of the Xi. Then {Yn : n = 1,2,3, . . .} is a martingale with respect to{Xn : n= 1,2,3, . . .}.
The martingale in Example1.34has some remarkable properties.Robbins (1970) showed that for any >1,
Pr(Yn≥for somen≥1)≤1/. (1.282) Robbins’s proof of (1.282) is straightforward. LetN be the first n≥1 such that Qni=1g(Xi)≥Qi=1n f(Xi), withN =∞ if no such noccurs. Also, let gn(t) =Qni=1g(ti) andfn(t) =Qni=1f(ti).
Pr(Yn≥for somen≥1) = Pr(N <∞) = ∞ X i=1 Z I{n}(N)fn(t)dt ≤ 1 ∞ X i=1 Z I{n}(N)gn(t)dt ≤ 1.
Another important property of the martingale in Example1.34is Yn
a.s.
→ 0. (1.283)
You are asked to show this in Exercise 1.83. Example 1.35 Bachelier-Wiener process
If {W(t) : t ∈ [0,∞[} is a Bachelier-Wiener process, then W2(t)
−t is a martingale. (Exercise.)
Example 1.36 A martingale that is not Markovian and a Markov process that is not a martingale
The Markov property is based on conditional independence of distributions
and the martingale property is based on equality of expectations. Thus it is easy to construct a martingale that is not a Markov chain beginning with
X0 has any given distribution with V(X0)>0. The sequence {Xt : EXt = EXt−1,VXt=Ptk=0−1VXk}is not a Markov chain.
A Markov chain that is not a martingale, for example, is {Xt : Xt d = 2Xt−1}, whereX0 has any given distribution with E(X0)6= 0.
A common application of martingales is as a model for stock prices. As a concrete example, we can think of a random variable X1 as an initial sum (say, of money), and a sequence of events in which X2, X3, . . .represents a sequence of sums with the property that each event is a “fair game”; that is, E(X2|X1) = X1 a.s.,E(X3|X1, X2) = X2 a.s., . . .. We can generalize this somewhat by lettingDn=σ(X1, . . . , Xn), and requiring that the sequence be such that E(Xn|Dn−1)
a.s. = Xn−1.
Doob’s Martingale Inequality
A useful property of submartingales is Doob’s martingale inequality. This inequality is a more general case of Kolmogorov’s inequality (B.11), page849, and the H´ajek-R`enyi inequality (B.12), both of which involve partial sums that are martingales.
Theorem 1.69 (Doob’s Martingale Inequality)
Let {Xt : t ∈[0, T]}be a submartingale relative to {Dt : t∈[0, T]} taking
nonnegative real values; that is, 0 ≤ Xs ≤ E(Xt|Dt) for s, t. Then for any
constant >0 andp≥1, Pr sup 0≤t≤T Xt≥ ≤1pE(|XT| p). (1.284) Proof.***fix
Notice that Doob’s martingale inequality implies Robbins’s likelihood ratio martingale inequality (1.282).
Azuma’s Inequality
extension of Hoeffding’s inequality (B.10), page848