In [10], an alternate approach to constructing almost invariant aggregates of a given stochastic matrix is presented. Here, we only present the algorithm, itself. In [24], we examine this algorithm in detail and present some supplemental results concerning its implementation.
A unitary matrix is a square complex matrix U such that U U∗ = I. A real unitary matrix evidently satisfies U UT = I; a real unitary matrix is referred to as an
orthogonal matrix. Let A be a n × n complex matrix. A singular value decomposition of A is an expression
A = U ΣV∗
where U and V are unitary matrices and Σ is a nonnegative diagonal matrix where the diagonal entries satisfy σii≥ σjj for all i < j. When A is real then U and V can
be taken to be orthogonal matrices, in which case we have
A = U ΣVT.
The ith columns of U and V are referred to as left and right singular vectors, re- spectively, of A associated with the singular value σii. If A is real and we let the ith
columns of U and V be ui and vi, respectively, we then have
Avi = σiiui and ATui = σiivi.
We label the singular values of A as σi(A) = σii. The number σ1(A) is, in fact, equal
to the euclidean 2-norm of A: σ1(A) = kAk2. See [14] for a thorough exposition of
the singular value decomposition.
The SVD-based algorithm is very simply expressed as a recursive algorithm. Its only input is a substochastic matrix A on a state space S. Within the algorithm there are references to singular vectors and coupling measures. It is up to the user to decide whether to utilise left or right-singular vectors and the π or 1-coupling measure; we will use w(E ) to represent this undetermined coupling measure of the set E . The output of the SVD-based algorithm is a partition {E1, . . . , Em} of S such
that w(Ek) > 1/2 for all k.
Algorithm 2 SVDA(A, S) if |S| = 1 then
return {S}
Terminate the algorithm. end if
Let v be a singular vector associated with σ2(A).
Let S+= {i ∈ S : vi ≥ 0}, S−= {i ∈ S : vi < 0}, A+= A(S+) and A−= A(S−).
if w(A+) ≤ 1/2 or w(A+) ≤ 1/2 then
return {S} else
return SVDA(A+, S+) ∪ SVDA(A−, S−)
end if
singular vectors rather than Fiedler vectors. An advantage of this approach is that singular vectors are, in general, more easily and reliably calculated than eigenvectors. In [15], a somewhat similar algorithm is presented. Rather than examining a singular vector associated with the second largest singular value of A, this algorithm proceeds by examining a singular vector associated with the second smallest singular value of I − A (where A is the matrix in question).
Chapter 4
The stochastic complement
We present the stochastic complement, which will be our primary tool for con- structing almost invariant aggregates of a given Markov chain. The stochastic comple- ment is introduced in [19]. It is there utilised as a tool for constructing the stationary distribution of a Markov chain and analysing the rate of convergence of a Markov chain to its stationary distribution (see Section 4.3). Many of the results of this chapter are discussed in [19], although some appear without proof.
4.1
Definition
Definition 4.1. Let A be a stochastic matrix with associated state space S. Let {C1, C2} be a partition of S and express
A ∼= A11 A12 A21 A22 ,
where Aij = A(Ci, Cj). If the matrix I − A22 is nonsingular, we define the stochastic
complement of C1 to be the matrix
S(C1) = A11+ A12(I − A22) −1
A21.
If I − A22 is singular, the stochastic complement of C1 is not defined.
Proposition 4.2. Let A be a stochastic matrix with state space S and let {C1, C2} be
a partition of S. Then, the stochastic complement S(C1) exists if and only if C2 does
not contain an essential class of states.
Proof Let B = A(C2). The matrix I − B is nonsingular if and only if 1 is not an
eigenvalue of B. By Corollary 2.7, we see that I − B is nonsingular if and only if C2
does not contain an essential class of states.
Remark. The stochastic complement can be seen as a way of removing the states C2 from the associated Markov chain. Proposition 4.2 tells us that we cannot remove
an entire essential class; i.e. the stochastic complement S(C1) exists if and only if C1
contains at least one member of every essential class.
Let A be a stochastic matrix and let
ω = i0 → i2 → · · · → it
be a directed walk of length t in the associated digraph. If t ≥ 1, the weight of ω is the product of the t transition probabilities along ω:
a(ω) = ai0i1ai1i2· · · ait−1it.
If ω is a walk of length 0, we define a(ω) = 1. The weight a(ω) is the probability of transitioning from i0 to it via the walk ω. By Proposition 1.5,
Atij = X
ω∈Ωij(t)
a(ω),
where Ωij(t) is the collection of directed walks from i to j with length equal to t. Let
ω be as above and let C ⊂ S be a subcollection of the state space; if
i1, . . . , it−1∈ C,
we refer to ω as a directed walk through C,
ω : i0 ;C it.
Note that the endpoints of a directed walk through C are not necessarily contained in C; such a walk is merely one in which every interior point is contained in C. Any directed walk of length 0 or 1 is trivially a walk through any collection, as it contains no interior points.
Proposition 4.3. Let A be a stochastic matrix and let C1 ⊆ S be a subcollection of
the state space; let C2 = S \ C1. If the stochastic complement S(C1) is defined, it is
1. the state space is equal to C1; and
2. for i, j ∈ C1, the transition probability sij is the sum of the weights of the directed
walks i;C2 j. Proof Let A ∼= A11 A12 A21 A22
(as in 4.1). We will preserve the indices of A in our examinations of its submatrices; for example, as long i ∈ C1 and j ∈ C2, (A12)ij = aij. The inverse of I − A22 (when it
exists) is nonnegative, via Lemma 2.8. The matrix
S(C1) = A11+ A12(I − A22) −1
A21
is entrywise nonnegative. Every row sum of A is 1 and so we have
A111 + A121 = 1 and A211 + A221 = 1;
these equalities in turn imply that
A111 = 1 − A121 and A211 = (I − A22) 1.
S(C1)1 = A111 + A12(I − A22) −1 A211 = (1 − A121) + A12(I − A22) −1 (I − A22) 1 = 1 − A121 + A121 = 1. Now, for i, j ∈ C1, let p
(t)
ij be the sum of the weights of the directed walks i;C2 j
with length equal to t. If t = 1, p(1)ij = aij; if t ≥ 2,
p(t)ij = P12P22t−2P21
ij.
Let pij be the sum of the weights of the directed walks i ;C2 j (of any length). Then,
pij = p (1) ij + p (2) ij + p (3) ij + p (4) ij + . . . = aij + (A12A21)ij + (A12A22A21)ij + (A12A222A21)ij + . . . = (A11+ A12(I + A22+ A222+ . . .) A21)ij = A11+ A12(I − A22) −1 A21 ij = sij.
Remark. The Markov chain described above has a straightforward interpretation (found in [19]). We observe the chains (in the original process) that have have x0 ∈ C1;
every time the process leaves C1 we imagine “fast-forwarding” until we return to C1,
ignoring any time spent in C2. That is, if a realization of the original Markov chain
is given by
x0, x1, x2, . . .
where x0 ∈ C1, the corresponding realization of the stochastic complement S(C1) is
x0, xt1, xt2, . . .
where
t1 = inf{t ≥ 1 : xt∈ C1}, t2 = inf{t ≥ t1+ 1 : xt∈ C1}, t3 = inf{t ≥ t2+ 1 : xt ∈ C1},
and so forth.
4.1.1
Stochastic complements of substochastic matrices
Let B be a substochastic matrix with state space C. Recall that an essential class of states with respect to B is a subset E ⊆ C such that B(E ) is irreducible and stochastic. Such a collection exists if and only if ρ(B) = 1, in which case B is not properly substochastic.
We define a stochastic complement of a substochastic matrix in exactly the same manner as for a stochastic matrix. That is, let B be a substochastic matrix with state space C and let {C1, C2} be a partition of C such that C2 does not contain an essential
class of states. Express
B ∼= B11 B12 B21 B22 ,
where Bij = B(Ci, Cj). Then, we define the stochastic complement of C1 to be the
S(C1) = B11+ B12(I − B22) −1
B21.
The stochastic matrices of order n are a subfamily of the substochastic matrices of order n. In the following section, we will prove a number of results concerning stochastic complements of substochastic matrices, with the understanding that these apply to stochastic complements of stochastic matrices.