Model Checking Complexity for LTL(F, G, ^, _)

Next we show that removing nexts (X) and untils (U ) from the logic makes the quali-tative model checking problem much easier. We prove that the model checking problem of MDPs against LTL(F, G,^, _) is in the complexity class NP. To do so we first take a closer look at the result by Courcoubetis and Yannakakis [19] which proposed the use of LDBAs for qualitative model checking.

Proposition 4.7. Given an MDP M and a limit-deterministic B¨uchi automaton A, the problem of checking if there exists S such that Pr^S_M(JAK) > 0, can be solved by taking a cross-product of M and A and checking if this product has a rechable BSCC containing a state (s, q) where q is a final state of A.

Checking for the existence of such a final BSCC boils down to doing a linear time graph analysis of the product. We have already seen how to transform LTL to LDBAs. The construction for ' 2 LTL(F, G, ^, _) produces an exponential sized LDBA A', so it would seem Proposition 4.7 is not useful for proving our desired NP upper bound. The key idea we introduce here is the following: the automaton A' can be split into a disjoint union of exponentially many LDBAs each of which is polynomially large in the size of '. In Proposition 4.7, if A is a disjoint union of multiple LDBAs, then the product of M and A has final BSCC if and only if the product ofM and some individual component of A has a final BSCC. The NP-algorithm guesses this individual component ofA (of polynomial size) and performs the graph analysis for the product of M and the indivudal component in time polynomial in both M and '.

In order to understand the splitting ofA^' we recall the core idea behind the construction of A' for ' 2 LTL(F, G, ^, _). For each F or G subformula of ', the automaton keeps

track of how often is true, which is one of three things:

1. is always true

2. is true at some point but not always 3. is never true

This yields a tri-partition, ⇡ =h ↵(⇡) | (⇡) | (⇡) i, of all the F, G subformulae of '. If a F subformula is in ↵, or we take it to mean that it is never true, true at some point but not always, or always true respectively. If G subformula is in ↵, or we take it to mean that it is always true, true at some point but not always, or never true respectively. With this semantics in mind we see that a subformula in ↵ or should remain in ↵ or respectively in the future, and a subformula in can remain in only for a finite time before moving to ↵. It turns out that, for a given input word w, correctly guessing (a) the triple ⇡ at the beginning of w and (b) the points along w at which subformulae move from to ↵, enables us to check if w satisfies the original formula '. A key observation here is that a triple at a certain point not only tells us how often a F, G subformula is true from that point onwards in the future, but also whether or not the subformula is true at that point. In other words, a triple refines the truth of F, G formulae. This observation is used to inductively check that the guessed triple is correct at every point. We encourage the reader to go back and refer Section4.1 for a detailed account of the construction. Recall that the state of the automaton for ' is of the form (⇡, k) where:

• ⇡ is a triple reflecting how often the F, G subformulae are true on the remaining input.

• k is an integer counter no larger than |'|, which is updated deterministically.

The transitions of the automaton allow moving from a state with ⇡ = h ↵ | | i to a state with ⇡⁰ = h ↵⁰| ⁰| ⁰i only if ↵ ✓ ↵⁰, ⁰ ✓ and = ⁰ (⇡ v ⇡⁰ for short) in accordance with the semantics we associate with the triple. That is ⇡ v ⇡⁰ is a necessary condition for a transition to move from ⇡ to ⇡⁰, i.e., subformulae in are allowed to move ↵ while the remaining stay put. In order to split this automaton into smaller components as anticipated earlier, we add restrictions to the order in which the formulae in are moved to ↵. First, let us

fix an initial triple ⇡ =h ↵0| 0| 0i. Given ⇡, consider a ranking function ⇢ : 0 ! N whose range is allowed to be any consecutive set of positive integers starting from 1, i.e. {1, . . . , n}.

Given ⇡ and ⇢ we are going to define a component A(⇡,⇢) of the original automaton A'. We define the space of possible triples ⇡i =h ↵ⁱ| ⁱ| ⁱi for i 2 {0, 1, . . . , n} as follows:

↵i ={ 2 ⁰ | f( )  i} [ ↵⁰ (4.5)

i ={ 2 ⁰ | f( ) > i} (4.6)

i = 0 (4.7)

The states of A(⇡,⇢) are those states of A' where the triple is restricted to be some ⇡_i as defined above. A transition ⌧ , say (⇡i, m) ! (⇡^j, n), is allowed in A^(⇡,⇢) i↵ ⌧ is a valid transition in A' and either j = i or j = i + 1. In A(⇡,⇢) a transition is allowed to either keep the triple unchanged (when j = i), or move only the formulae mapped to i + 1 from to ↵ (when j = i + 1). Thus the ranking function ⇢ restricts the order in which the subformulae move from to ↵. A subformula with smaller rank is moved earlier compared to one with a larger rank. Note that two or more formulae can be mapped to the same number, which means those formulae are moved simultaneously. The initial state of A^(⇡,⇢) is defined to be (⇡0, 0) and the state (⇡n, 0) is marked as the only final state. Note that the size of the automaton A^(⇡,⇢) is n +| ⁰| which is linear in |'|. The number of di↵erent (⇡, ⇢) is exponential in ', hence there can be exponentially many di↵erent individual components.

What remains to be seen is that the disjoint union of these components U

A^(⇡,⇢) accepts exactly the same language as A'. Since A(⇡,⇢) is a projection of A' it is the case that JA^(⇡,⇢)K ✓ JA^'K and so J]A^(⇡,⇢)K ✓ JA^'K. To see the other direction consider any word w accepted by A', and let (⇡0, k0), (⇡1, k1), . . . be an accepting run for w on A' with ⇡i = h ↵ⁱ| ⁱ| ⁱi. From the construction of A^' we know that ⇡0 v ⇡¹ v ⇡²· · · . Identify all the positions j1 < j2 <· · · < jn where the triple changes, i.e.,

(⇡0 = ⇡1· · · = ⇡j1)@ (⇡j1+1 =· · · = ⇡j2)@ (⇡j2+1 =· · · = ⇡j3)@ (· · · ) @ (⇡jn =· · · (4.8) Here ji is the i^th time the triple changes, n being the last. Now we consider the automaton A(⇡0,⇢) where ⇢( ) ^def= i if moves from to ↵ at position ji, i.e., 2 ji and 2 ↵ji+1. Observe that the above accepting run is also an accepting run ofA^(⇡,⇢) on the word w. This gives us JA^'K ✓ J]A^(⇡,⇢)K.

s

Figure 4.4: Markov chain reduction for a boolean formula having n variables.

Thus we have succesfully split A^' into exponentially many individual components of linear size. The index (⇡, ⇢) for any component requires only polynomially many bits to represent. This combined with our earlier observation of using Proposition4.7for the disjoint union gives us the NP-algorithm for qualitative model checking against LTL(F, G,^, _).

NP-hardness: Next we show that the qualitative model checking problem against the fragment LTL(F,^) is NP-hard. We show this lower bound not just for MDPs but for Markov chains. We reduce the problem of boolean satisfiability to qualitative model checking of Markov Chains. Consider a boolean formula n negation normal form which consits of variables x₁, . . . , x_n. We construct a Markov chain Mn as shown in Figure 4.4, which has start state s, a sink state t, and states q1, . . . , qn and r1, . . . , rn where n is the number of variables in . s goes to q₁ or r₁ with half probablity each. Each q_i, r_i (for i < n) has transitions to qi+1, ri+1 with half probability each, and qn, rn proceed to t and remain there with probability 1. Next we fix the set of propositions {y1, . . . , y_n, z₁, . . . , z_n} which we will use to label the Markov chain. Each yi is assigned to be true at and only at qi and each zi

is set to true at and only at r_i. Next we transform the formula into a LTL(F,^) formula

0 by replacing the positive literals xi with Fyi and the negative literals ¬xi with Fzi. For example (x₁ ^ ¬x2) _ x3 would become (Fy₁ ^ Fz2) _ Fy3. Now the claim is that is satisfiable if and only if ⁰ has a non-zero probability of being satisfied in Mn. There is a one to one correspondence between assignments to variables in and paths from s to t in Mn such that an assignment satisfies i↵ the trace generated by the corresponding path satisfies ⁰. Every path from s to t has a non-zero probability of occuring, and therefore being satisfiable is equivalent to ⁰ being satisfied in Mn with non-zero probability. This completes the reduction showing that qualitative model checking is NP-hard for LTL(F,^).

LTL Fragment Qualitative Model checking LTL(F,^)

NP-complete LTL(F, G,^, _)

LTL(F, X,^)

EXPTIME-complete LTLD

Table 4.2: Summary of results: qualitative model checking complexity of MDPs against various fragments.

4.6 Conclusion

In this chapter we considered the qualitative model checking problem for MDPs against LTL specifications. We used the automata theoretic approach to solving the problem, which uses limit deterministic automata. We showed efficient constructions of such automata which improve upon existing constructions by an exponential factor. This results in improving the upper bound from 2EXPTIME to EXPTIME for a large class of properties, namely LTLD. We also showed that the problem is EXPTIME-complete for LTLD(and LTL(F, X,^)). We also showed that the automata we obtain for LTL(F, G,^, _), using the same translation, can be appropriately split so that they can be used in an NP algorithm for qualitative model checking, and we showed that is the best we can achieve by proving matching lower bound.

These results are summarized in Table 4.2. There are some questions that still remain open.

Is it possible to have an exponential sized limit deterministic translation for a logic bigger than LTLD? This is an open question. But we do know from [52] that there is an double exponential lower bound for the full logic.

Chapter 5

Quantitative Model Checking

In this chapter we present new upper bounds for quantitative model checking of MDP for various fragments of LTL. We also prove matching lower bounds for all these fragments.

The fragments we consider will be restricting the use of temporal operators to G, F and X.

The upper bounds we prove follow the automata theoretic approach but in a manner which di↵ers from what we have seen previously. Here we rely on being able to construct B¨uchi automaton for the LTL formula, but instead of explicitly constructing the automa-ton from the given formula, we construct parts (states and transitions) of the automaautoma-ton implicitly. The algorithm analyzes the product of the MDP and the automaton, without constructing the entire product, but constructing it on a need-to-know basis. This analysis calculates probabilities of repeatedly reaching a set of states in an MDP. The algorithm for doing this calculation is the core technical result that is used in all the upper bounds in this chapter. The lower bounds for these problems are obtained by adapting the lower bounds for solving 2-player games with LTL objectives.