The PCTL Until Operator for DTMCs

As our first example, we consider an MTBDD implementation of the numerical compu-

tation for the PCTL until operator, P./ p[φ1 U φ2], over DTMCs. This algorithm was

discussed in Section 3.3.1. The MTBDD version is given in Figure 5.4. It is essentially

the same as the one proposed in [BCHG+97]. The probabilities are computed by the

MTBDD algorithm PctlUntil. It takes as input two BDDs, phi1 and phi2, representing

the sets of states Sat (φ1) and Sat (φ2), respectively, and returns an MTBDD represent-

ing the vector of probabilities ps(φ1 U φ2) for each state s. It also uses the MTBDD P

representing the transition probability matrix of the DTMC.

The first step (lines 1–3) determines the sets Sno_{, S}yes _{and S}?_{. This uses the pre-}

computation algorithms Prob0 and Prob1, the implementation of which was described in the previous section. Secondly (lines 4–6), the linear equation system A · x = b is constructed, as described in Section 3.3.1. The matrix A and vector b are represented by

MTBDDs A and b respectively. We first build the matrix P0 (represented by P0) which

is equal to the matrix P but with the rows corresponding to states not in S? _{set to zero.}

PctlUntil(phi1, phi2) 1. sno:= Prob0(phi1, phi2) 2. syes:= Prob1(phi1, phi2, sno) 3. s?:= ¬ (sno∨ syes) 4. P0:= s?× P 5. _{A := Identity(x, y) − P}0 6. b := syes 7. _{probs := SolveJacobi(A, b, b)} 8. return probs

Figure 5.4: The PctlUntil algorithm

latter representing S?_{. The matrix A = I − P}0

is then constructed using the Identity and Apply functions. In line 7, the solution of the linear equation system A · x = b is computed, using the Jacobi iterative method. This constitutes the bulk of the work and is contained in a separate algorithm, SolveJacobi.

As stated in Section 3.3, we use iterative methods for solving linear equation systems, rather than alternative, direct methods such as Gaussian elimination or L/U decomposi- tion. This is because we are aiming to study large probabilistic models, which will produce very large linear equation systems. Direct methods usually require modifications to the matrix A, which are costly both in terms of space and time.

This argument applies regardless of the data structure being used. In our case, how-

ever, it is particularly relevant. Work by [BFG+_{93] showed that MTBDDs are very poorly}

suited to methods such as Gaussian elimination. As we saw in the previous chapter, the effectiveness of MTBDDs relies heavily on them being used to store regular, structured information. Modifications to the matrix A, such as those made by Gaussian elimination, inevitably lead to a significant loss in regularity and a consequent blow-up in the size of the MTBDD. Furthermore, the operations required to perform these modifications work on individual elements, rows and columns of the matrix. These are particularly difficult to implement on inherently recursive data structures such as MTBDDs. The iterative meth- ods we use, on the other hand, do not modify the matrix throughout the computation and can be implemented with matrix-vector multiplication, for which efficient MTBDD algorithms exist.

The problem of implementing iterative solution methods using MTBDDs was first considered in [HMPS94], which implemented steady-state probability calculation using the Power method. [HMKS99] extended this work, also presenting MTBDD algorithms for the Jacobi and Gauss-Seidel methods. In Figure 5.5, we give the function SolveJacobi, the MTBDD implementation of the Jacobi method.

SolveJacobi(A, b, init)

1. _{d := Abstract(max, y, A × Identity(x, y))} 2. A0_{:= A × Const(−1) × ¬Identity(x, y)} 3. sol := init

4. done := false

5. while (done = false)

6. sol0 _{:= MVMult(A}0_{, sol)} 7. sol0 := sol0+ b

8. sol0 := sol0÷ d

9. _{if (MaxDiff(sol, sol}0) < ε) then 10. done := true

11. endif 12. sol := sol0 13. endwhile 14. return sol

Figure 5.5: The SolveJacobi algorithm

The algorithm can be compared to the description of the Jacobi method we gave in Section 3.5. Note that there were two alternatives presented there: one expressed in terms of operations on individual matrix elements; and one in terms of matrix-vector multiplication. We select the latter because, as described above, it is far more efficient to

implement in MTBDDs. The first three lines of Figure 5.5 set up the MTBDDs A0, d and

sol which will be used in the main iterative loop. In terms of the description of the Jacobi

method in Section 3.5.1, A0 corresponds to the matrix L + U and d stores the diagonal

values from the matrix D. The MTBDD sol represents the solution vector.

The main part of SolveJacobi is the loop in lines 4–13. Each iteration computes

the next approximation to the solution vector, sol0, from the previous one, sol. This is

done with one matrix-vector multiplication and a pointwise addition and division on a

vector. Each iteration also contains a convergence check which compares sol and sol0 to

determine whether or not the method should be terminated. Various stopping criteria can be used, as discussed in Section 3.5. We check if the maximum relative difference between elements of the two vectors is below some threshold ε. We assume the presence

of a function MaxDiff(v1, v2) which computes this difference between two vectors rep-

resented by MTBDDs v1 and v2. This could be done with basic MTBDD operations, e.g.

FindMax(Abs((sol0−sol)÷sol0)). In fact, there are also built-in operations in the CUDD

package which can be used to compute this directly from the MTBDDs.

The JOR method can be implemented as a simple modification of the Jacobi method (see Appendix D for the exact details). To encode Gauss-Seidel though, or the related SOR

PctlUntilMax(phi1, phi2) 1. sno:= Prob0A(phi1, phi2) 2. syes:= Prob1E(phi1, phi2) 3. s?:= ¬ (sno∨ syes)

4. Steps0:= s?× Steps 5. probs := syes 6. done := false

7. while (done = false)

8. probs0_{:= MVMult(Steps}0, probs) 9. probs0_{:= Abstract(max, z, probs}0) 10. probs0:= probs0+ yes

12. _{if (MaxDiff(probs, probs}0) < ε) then 13. done := true

14. endif

15. probs := probs0 16. endwhile

16. return probs

Figure 5.6: The PctlUntilMax algorithm

method, is more difficult. For efficiency reasons, we must rely on the matrix formulation of the method given in Section 3.5, which is again based on matrix-vector multiplication. The implementation of this in MTBDDs is considered in [HMKS99] but the need to compute a matrix inverse and the amount of extra work this entails make it an unattractive option. This is unfortunate, because these two methods usually require significantly less iterations to converge, which could have a marked effect on the overall time required for model checking.