Distribution Based Semantics
7.2 Summary of Results for MDPs
In this Section we will briefly go through the results for problems pertaining to MDPs.
These results follow from some well known results on probabilistic finite automata (PFA) which we shall touch upon. Most of our contribution in this thesis will be results regarding Markov chains, and we provide the results for MDPs here to show a contrast between the complexities.
7.2.1 Undecidability
The first result we observe is that the model checking problem for MDPs even when the labeling are restricted to be binary is undecidable. This result relies on the undecidability of the emptiness checking of probabilistic finite automata (PFA) by Condon and Lipton [17].
Therefore we review the definitions related to PFAs.
A PFA is a tuple A = (Q, ⌃, ( ) 2⌃, µ0, F ), where Q is a finite set of states, ⌃ is the finite input alphabet, F ✓ Q is the set of final states and µ0 2 Dist(Q) is the initial distribution and ( ) 2⌃ is an indexed set of stochastic matrices with dimension |Q| ⇥ |Q|. For a symbol a, a(s, t) represents the probability of going from state s to t on input symbol a. For any input word w 2 ⌃⇤ of length n the probability of going from s to t along w = a1a2· · · an is then given by w(s, t) where w is the matrix ( a1 · a2· · · an). The distribution reached on input w 2 ⌃⇤ in A is then given by µ0 w.
The acceptance probability of a word w 2 ⌃⇤ on PFA A is given by P
q2F(µ0 w)(q) or µ0 w⌘F where ⌘F is the column vector such that ⌘F(j) = 1 if j 2 F and ⌘F(j) = 0 otherwise.
We will say that ⌘F is the vector corresponding to F.
Given a cut-point t 2 [0, 1], the language of F w.r.t cut-point t is defined as the set L>t(A) = {w 2 ⌃⇤ | µ0 w⌘F > t}. The result by Condon and Lipton [17] is that the emptiness problem of PFAs for any non-zero cut-point is undecidable.
Theorem 7.1 (Condon and Lipton [17]). Given a PFA A the problem of checking if L>1
2(A) =; is undecidable.
Using this we show the undecidability of the model checking problem:
Theorem 7.2. The problem of model checking a MDP M with respect to a binary labeling function is undecidable.
Proof. We reduce the emptiness problem for PFAs to our model checking problem. Let F = (Q, ⌃, ( ) 2⌃, µ0, F ) be a PFA over alphabet ⌃. We will construct a MDP M = (Q, Act, , µ0) which has the same state space asF, the set of actions Act = ⌃, the transition function (q, a) is the qth row of a, and the initial distribution µ0 is the same. Observe that
if there is a finite path of M, µ0µ1. . . µn obtained from the schedule w = w0w1. . . wn then, the word w is accepted with probability µn(F ) = µ0 w⌘F byF, and vice versa.
Let AP = {p} be the singleton proposition (label), and consider the labeling function such that p 2 (µ) i↵ P
q2Fµ(q) > 12. Such a labeling is indeed binary and from the observation of the previous paragraph we obtain that L>1
2(F) is empty i↵ M has a path that reaches a distribution labeled p.
7.2.2 Decidability under Robustness
Next, we observe that model checking MDPs becomes decidable when we restrict the binary labeling to be robust. We make use of the celebrated result by Rabin [48] which proves languages accepted by PFAs with isolated cut-points is regular. A cut-point t is said to be ✏-isolated if there exists an ✏ > 0 such that for all w 2 ⌃⇤,|µ0 w⌘F t| > ✏.
Theorem 7.3 (Rabin [48]). Given a PFA F with n states and r final states, and a cut-point t which is ✏-isolated, the language L>t(F) is regular and is recognized by a DFA with (1 + (r/✏))(n 1) states.
Theorem 7.4. The problem of model checking a MDP M with respect to a robust binary labeling where the degree of robustness ✏ is given is solvable in EXPTIME.
Proof. Let a particular p2 AP be parameterized by a1, . . . , an, b for the binary set it repre-sents. Now we can define a PFA over the alphabet Act, where the states and transitions are given by the MDP M, final states are those whose coefficients ai are 1, and the cut-point for the PFA is b. Observe that a word w 2 Act⇤ is accepted by the PFA i↵ the distribution that the MDP reaches via w is such that p holds true on it according to . Now since is robust, we have that the PFA is isolated. Using Theorem 7.3 we can construct an ex-ponential sized DFA that accpets the same language as the PFA. The DFA accepts a word w2 Act⇤ i↵ p2 (µ0 w). Therefore the synchronous product of these DFAs gives us a Moore machine B which outputs the labels according to along any infinite word in Act!. Taking a cross product of B with the B¨uchi automaton A of the specification where the output of the Moore machine B is fed into A gives us an exponential sized B¨uchi automaton A ⇥ B.
The automatonA ⇥ B over Act! accepts all and only all violating schedules. Model checking
then reduces to checking emptiness of which can be done time linear in the size of the graph.
This gives us the EXPTIME upper bound.
7.2.3 Undecidability of Checking Robustness
The problem of checking if a binary label is robust is equivalent to the problem of checking isolation of PFAs. Given MDPM = (Q, Act, , µ0) and a proposition p in a binary label where Up is parameterized by a1, a2, . . . , an, b, we construct a PFA A = (Q, ⌃, ( ) 2⌃, µ0, F ) as before where Q and µ0 are the same, ⌃ = Act, is the stochastic matrix whose qth row is (q, ), and F = {q 2 Q | aq = 1}. A along with cut-point b is then isolated i↵ the proposition p is robust for M. Simlarly given a PFA A and cut-point b one can construct a MDP and a labeling such that the two problems are identical. The isolation problem turns out to be undecidable, which gives us the undecidability of the robustness problem.
Theorem 7.5 ([9,25,11]). Given PFA F and cut-point t, the problem of deciding whether t is isolated for F is ⌃02-complete (undecidable).
Theorem 7.6. The problem of checking whether a given proposition is robust w.r.t binary label and MDP M is ⌃02-complete.
Corollary 7.1. The problem of checking whether a given proposition is limit robust w.r.t binary label and MDP M is undecidable.
7.2.4 Roadmap
In the remaining chapters we consider the model checking and the robustness problems for Markov Chains. First we prove that the problem of checking limit robustness of a given binary label for Markov Chains is coNP-complete. We also prove that the robustness checking problem for a binary label for Markov Chains is coNPRP. Both these results are covered in Chapter 8. Regarding the model checking problem, we prove that it is decidable in PSPACE for Markov chains against binary labels that are limit robust in Chapter 9.