Classical (and Useful) Markov Chains
2.6. RANDOM WALKS ON GROUPS
We write ˜E for expectations for this new chain. By the proof of Proposition 2.8, the stationary probability ˜πof the truncated chain is given by
˜
πk =
wk
Pℓ
j=0wj
for 0≤k≤ℓ. Since in the truncated chain the only possible moves fromℓare to stay put or to step down toℓ−1, the expected first return time ˜Eℓ(τℓ+) satisfies
˜ Eℓ(τℓ+) = (rℓ+pℓ)·1 +qℓ ˜ Eℓ−1(τℓ) + 1 = 1 +qℓE˜ℓ−1(τℓ). (2.14) By Proposition1.14(ii), ˜ Eℓ(τℓ+) = 1 ˜ π(ℓ) = 1 wℓ ℓ X j=0 wj. (2.15)
We have constructed the truncated chain so that ˜Eℓ−1(τℓ) =Eℓ−1(τℓ). Rearranging
(2.14) and (2.15) gives Eℓ−1(τℓ) = 1 qℓ ℓ X j=0 wj wℓ − 1 = 1 qℓwℓ l−1 X j=0 wj. (2.16)
To findEa(τb) fora < b, just sum:
Ea(τb) = b
X
ℓ=a+1
Eℓ−1(τℓ).
Consider two important special cases. Suppose that (pk, rk, qk) = (p, r, q) for 1≤k < n,
(p0, r0, q0) = (p, r+q,0), (pn, rn, qn) = (0, r+p, q)
for p, r, q≥ 0 with p+r+q = 1. First consider the case where p6=q. We have
wk= (p/q)k for 0≤k≤n, and from (2.16), for 1≤ℓ≤n,
Eℓ−1(τℓ) = 1 q(p/q)ℓ ℓ−1 X j=0 (p/q)j = (p/q) ℓ −1 q(p/q)ℓ[(p/q)−1] = 1 p−q " 1− q p ℓ# .
Ifp=q, thenwj = 1 for allj and
Eℓ−1(τℓ) = ℓ
p.
2.6. Random Walks on Groups
Several of the examples we have already examined and many others we will study in future chapters share important symmetry properties, which we make explicit here. Recall that agroupis a setGendowed with an associative operation
·:G×G→Gand anidentity id∈Gsuch that for allg∈G, (i) id·g=g andg·id =g.
28 2. CLASSICAL (AND USEFUL) MARKOV CHAINS
Given a probability distribution µ on a group (G,·), we define the random walk onG with increment distributionµas follows: it is a Markov chain with state space G and which moves by multiplying the current state on the left by a random element ofG selected according toµ. Equivalently, the transition matrix
P of this chain has entries
P(g, hg) =µ(h) for allg, h∈G.
Remark 2.9. We multiply the current state by the incrementon the left be- cause it is generally more natural in non-commutative examples, such as the sym- metric group—see Section 8.1.3. For commutative examples, such as the two de- scribed immediately below, it of course does not matter on which side we multiply.
Example 2.10 (Then-cycle). Let µ assign probability 1/2 to each of 1 and
n−1≡ −1 (modn) in the additive cyclic groupZn={0,1, . . . , n−1}. Thesimple
random walk on then-cycle first introduced in Example1.4is the random walk onZn with increment distributionµ. Similarly, let ν assign weight 1/4 to both 1
andn−1 and weight 1/2 to 0. Thenlazy random walk on then-cycle, discussed in Example1.8, is the random walk onZn with increment distributionν.
Example2.11 (The hypercube). The hypercube random walks defined in Sec- tion2.3are random walks on the groupZn
2, which is the direct product ofncopies
of the two-element groupZ2 ={0,1}. For the simple random walk the increment
distribution is uniform on the set{ei : 1≤i≤n}, where the vectoreihas a 1 in the
i-th place and 0 in all other entries. For the lazy version, the increment distribution gives the vector0(with all zero entries) weight 1/2 and eachei weight 1/2n.
Proposition 2.12. Let P be the transition matrix of a random walk on a finite groupGand let U be the uniform probability distribution on G. ThenU is a stationary distribution for P.
Proof. Let µ be the increment distribution of the random walk. For any
g∈G, X h∈G U(h)P(h, g) = 1 |G| X k∈G P(k−1g, g) = 1 |G| X k∈G µ(k) = 1 |G| =U(g).
For the first equality, we re-indexed by settingk=gh−1. 2.6.1. Generating sets, irreducibility, Cayley graphs, and reversibil- ity. For a setH ⊂G, lethHibe the smallest group containing all the elements of
H; recall that every element ofhHican be written as a product of elements inH
and their inverses. A setH is said togenerate GifhHi=G.
Proposition 2.13. Let µ be a probability distribution on a finite group G. The random walk on G with increment distribution µ is irreducible if and only if
S={g∈G : µ(g)>0} generates G.
Proof. Letabe an arbitrary element ofG. If the random walk is irreducible, then there exists an r > 0 such that Pr(id, a) > 0. In order for this to occur,
there must be a sequences1, . . . , sr∈Gsuch thata=srsr−1. . . s1 andsi ∈S for
i= 1, . . . , r. Thusa∈ hSi.
Now assumeS generatesG, and considera, b∈G. We know thatba−1can be
2.6. RANDOM WALKS ON GROUPS 29
has finite order, any inverse appearing in the expression forba−1 can be rewritten
as a positive power of the same group element. Let the resulting expression be
ba−1=s
rsr−1. . . s1, wheresi∈S fori= 1, . . . , r. Then
Pm(a, b)≥P(a, s1a)P(s1a, s2s1a)· · ·P(sr−1sr−2. . . s1a,(ba−1)a)
=µ(s1)µ(s2). . . µ(sr)>0.
WhenSis a set which generates a finite groupG, thedirected Cayley graph associated toGandS is the directed graph with vertex setGin which (v, w) is an edge if and only ifv=swfor some generators∈S.
We call a set S of generators of G symmetric if s ∈ S implies s−1
∈ S. WhenS is symmetric, all edges in the directed Cayley graph are bidirectional, and it may be viewed as an ordinary graph. When Gis finite and S is a symmetric set that generates G, the simple random walk (as defined in Section 1.4) on the corresponding Cayley graph is the same as the random walk on Gwith increment distributionµtaken to be the uniform distribution onS.
In parallel fashion, we call a probability distributionµon a groupGsymmetric ifµ(g) =µ(g−1) for everyg∈G.
Proposition2.14. The random walk on a finite group Gwith increment dis- tribution µis reversible if µis symmetric.
Proof. LetU be the uniform probability distribution onG. For anyg, h∈G, we have that U(g)P(g, h) = µ(hg −1) |G| and U(h)P(h, g) = µ(gh−1) |G|
are equal if and only ifµ(hg−1) =µ((hg−1)−1). Remark 2.15. The converse of Proposition2.14is also true; see Exercise2.7.
2.6.2. Transitive chains. A Markov chain is called transitive if for each pair (x, y)∈Ω×Ω there is a bijectionϕ=ϕ(x,y): Ω→Ω such that
ϕ(x) =y and P(z, w) =P(ϕ(z), ϕ(w)) for all z, w∈Ω. (2.17) Roughly, this means the chain “looks the same” from any point in the state space Ω. Clearly any random walk on a group is transitive; setϕ(x,y)(g) =gx−1y. However,
there are examples of transitive chains that are not random walks on groups; see
McKay and Praeger (1996).
Many properties of random walks on groups generalize to the transitive case, including Proposition2.12.
Proposition2.16. LetP be the transition matrix of a transitive Markov chain on a finite state spaceΩ. Then the uniform probability distribution onΩis station- ary forP.
Proof. Fixx, y∈Ω and let ϕ: Ω→Ω be a transition-probability-preserving bijection for whichϕ(x) =y. LetU be the uniform probability on Ω. Then
X z∈Ω U(z)P(z, x) =X z∈Ω U(ϕ(z))P(ϕ(z), y) =X w∈Ω U(w)P(w, y),
30 2. CLASSICAL (AND USEFUL) MARKOV CHAINS
where we have re-indexed with w=ϕ(z). We have shown that when the chain is started in the uniform distribution and run one step, the total weight arriving at each state is the same. SincePx,z∈ΩU(z)P(z, x) = 1, we must have
X
z∈Ω
U(z)P(z, x) = 1
|Ω| =U(x).
2.7. Random Walks onZ and Reflection Principles
A nearest-neighbor random walk on Z moves right and left by at most
one step on each move, and each move is independent of the past. More precisely, if (∆t) is a sequence of independent and identically distributed {−1,0,1}-valued
random variables andXt=Pts=1∆s, then the sequence (Xt) is a nearest-neighbor
random walk with increments (∆t).
This sequence of random variables is a Markov chain with infinite state space
Zand transition matrix
P(k, k+ 1) =p, P(k, k) =r, P(k, k−1) =q,
wherep+r+q= 1.
The special case wherep=q= 1/2, r= 0 is the simple random walk onZ, as
defined in Section1.4. In this case
P0{Xt=k}= ( t t−k 2 2−t ift−kis even, 0 otherwise, (2.18)
since there are t−tk
2
possible paths of lengthtfrom 0 tok.
Whenp=q= 1/4 andr= 1/2, the chain is the lazy simple random walk onZ.
(Recall the definition of lazy chains in Section1.3.)
Theorem 2.17. Let(Xt)be simple random walk onZ, and recall that
τ0= min{t≥0 : Xt= 0}
is the first time the walk hits zero. Then Pk{τ0> r} ≤
12k
√r (2.19)
for any integersk, r >0.
We prove this by a sequence of lemmas which are of independent interest.
Lemma2.18 (Reflection Principle). Let (Xt)be either the simple random walk
or the lazy simple random walk onZ. For any positive integers j,k, andr, Pk{τ0< r, Xr=j}=Pk{Xr=−j} (2.20)
and
Pk{τ0< r, Xr>0}=Pk{Xr<0}. (2.21)
Proof. By the Markov property, the walk “starts afresh” from 0 when it hits 0, meaning that the walk viewed from the first time it hits zero is independent of its past and has the same distribution as a walk started from zero. Hence for any
s < randj >0 we have