As we saw in Section 3.3, model checking for several operators of PCTL and CSL reduces to the solution of a system of linear equations. We can assume that this system is of the traditional form A · x = b, where A is a real-valued matrix, b is a real-valued vector, and x is a vector containing the solution to be determined. Since the systems we solve are derived from model checking problems, where matrices and vectors are indexed over a set of states S, we assume that matrices and vectors are of size |S| × |S| and |S|, respectively, and use the indexing 0, . . . , |S|−1.
Solving a system of linear equations is, of course, a well known and much studied problem. Typically, methods for their solution fall into two distinct classes: direct methods and iterative methods. Direct methods compute the exact solution (within the bounds of numerical error) in a fixed number of steps. Examples include Gaussian elimination and L/U decomposition. Iterative methods compute successive approximations to the solution, terminating when the sequence of solutions has converged to within some pre- specified accuracy. Examples include the Power, Jacobi and Gauss-Seidel methods.
In this work we will only consider iterative methods. As with many applications, probabilistic model checking typically produces systems where A is large and sparse. Direct methods are not well suited to solving such systems because of a phenomenon known as fill-in. Modifications made to A as the computation progresses usually increase the number of non-zero elements. For large systems, the resulting growth in required storage space can make the computation infeasible. Fortunately, this is not the case for iterative methods. With the exception of some possible initialisation steps, the matrix is not modified at all during computation, and hence there is no increase in memory consumption.
There are also implications for the data structures used to implement the solution methods. For iterative techniques, we are free to select matrix storage schemes with strengths such as compactness and ease of access: we need not concern ourselves with the efficiency of modifications to the matrix. Conventionally, this argument is applied to sparse matrices, the traditional storage scheme for iterative methods, which we review in Section 3.6. The arguments will also apply, however, to the symbolic data structures, based on MTBDDs, which we introduce in Chapter 6.
In the following paragraphs we describe four of the most common iterative methods for solving systems of linear equations: Jacobi, Gauss Seidel, JOR and SOR. The general procedure for all four methods is as follows. Starting with some initial estimate, each iteration produces an increasingly accurate approximation to the solution of the linear
equation system. The approximation computed in the kth iteration is denoted x(k). We
refer to this as the solution vector or iteration vector. Using the same notation, we denote the initial estimate as x(0).
Each estimate x(k) is computed from the previous one x(k−1). The iterative process is
stopped when the solution vector is judged to have converged sufficiently. The criteria for convergence can vary. One approach is to stop when the maximum difference between elements of consecutive vectors is less than some threshold ε:
maxi
x(k)(i) − x(k−1)(i) < ε
One potential problem with, though, is that if the solution vector contains very small values (i.e. less than ε), the process may be terminated prematurely. A better approach, and the one we adopt in this thesis, is to check the relative difference between elements:
maxi
|x(k)(i) − x(k−1)(i)|
|x(k)(i)|
< ε
3.5.1
The Jacobi Method
The Jacobi method is based on the observation that the ith equation of the linear equation system A · x = b:
|S|−1
X
j=0
A(i, j) · x(j) = b(i)
can be rearranged as:
x(i) = b(i) −X
j6=i
A(i, j) · x(j) !
/A(i, i)
On the basis of this, in the Jacobi method, the ith element of the kth iteration vector is computed from the elements of the (k − 1)th vector as:
x(k)(i) := b(i) −X
j6=i
A(i, j) · x(k−1)(j) !
/A(i, i)
Note that for the systems which arise from PCTL or CSL model checking, the diagonal elements A(i, i) will always be non-zero. For our purposes, it is also useful to express a single iteration of the Jacobi method in terms of matrices and vectors, rather than individual elements:
x(k) := D−1· (L + U) · x(k−1)+ b
where D−(L+U) is a partitioning of the matrix A into its diagonal, lower-triangular, and upper-triangular elements respectively, i.e. D contains all the diagonal entries of A and L + U contains all the non-diagonal entries, negated. In this setting, the main operation required to perform a single iteration is a matrix-vector multiplication. We will see why this is useful when we implement symbolic versions of these algorithms in Chapters 5 and 6. Note also that the Jacobi method requires two vectors to be stored: one for the previous iteration’s vector, and one for the current one.
3.5.2
The Gauss-Seidel Method
The Jacobi method can be improved by observing that the kth update to the ith vector
element can actually use the new value x(k)(j) rather than the old value x(k−1)(j) for
j < i. This gives rise to the Gauss-Seidel method:
x(k)(i) := b(i) −X j<i A(i, j) · x(k)(j) −X j>i A(i, j) · x(k−1)(j) ! /A(i, i)
This usually converges much faster than the Jacobi method. Furthermore, it only requires a single vector to be stored: after each new vector entry is computed, its old value is no longer required and can be overwritten.
As with the Jacobi method, we can show that one iteration of the Gauss-Seidel method can be performed using matrix-vector multiplication:
x(k) := (D − L)−1· (U · x(k−1)+ b)
where D, L and U are as for the Jacobi method above. In this case, though, it may not be a sensible option since the computation of the inverse of (D − L) may require significant extra work or result in fill-in. Furthermore, with this formulation, we again need to store two vectors rather than one.
3.5.3
Over-Relaxation Methods
Both the Jacobi and Gauss-Seidel methods can be improved with a technique called over-relaxation. In an iteration, the new value of each vector element is first computed as described above, and then subjected to a weighted average between itself and the corresponding vector element from the previous iteration. The weights for this average are determined according to a parameter ω. For example, using the Jacobi method as a basis, we get:
x(k)(i) := (1 − ω) · x(k−1)(i) + ω · b(i) −X
j6=i
A(i, j) · x(k−1)(j) !
/A(i, i)
This is known as the JOR (Jacobi Over-Relaxation) method. The same technique can be applied to the Gauss-Seidel method, in which case it is referred to as the SOR (Successive Over-Relaxation) method. The idea is that the methods converge faster for a well chosen value of ω. It is known that JOR and SOR will only converge for ω ∈ (0, 2). Unfortunately, it is usually either impossible or impractical to compute the optimum value of ω. Typically, one must rely on heuristics to select a good value. The interested reader is referred to, for example [Ste94], for more details. Note also that, technically, for ω < 1, this technique should be referred to as under-relaxation, but for convenience it is often still known as over-relaxation.