2.3 Specifications for probabilistic models
2.3.1 Reachability properties
Many important problems when analyzing probabilistic models as presented in Section 2.2 can be reduced to so-called reachability properties. We are interested in a quantitative analysis such as:
“What is the probability to reach a certain set of states T starting in state s?”
Such a set of target states T might, e. g., represent a bad event in the model at hand and should only occur with a certain small probability. Formally, we formulate properties like P\λ(◊target)
for \ ∈ {<, ≤, =, >, ≥}, λ ∈ [0, 1] ⊆ R, target ∈ AP and T = {s ∈ S | target ∈ L(s)}. Note that in fact λ is from [0, 1] ∩ Q as no real-valued probability bound can be used in practice. P measures the probability for the property and is formalized later. For example, P≤0.1(◊target)
means that the probability of reaching a state labeled withtargethas to be less than or equal to 0.1. If the probability is higher at state s, this property evaluates tofalsefor this state. We also directly write P\λ(◊T ) for these properties, thereby implying that the set T consists exactly of
the states that are labeled withtarget. Moreover, we restrict the properties to the form P≤λ(◊T )
and sometimes P<λ(◊T ). The cases ≥ and > can be reduced to ≤ and < using negation and the
complement probabilities, e. g., P>λ(◊T ) is equivalent to P≤1−λ(◊¬T ).
In the following we explain how such properties can be verified for DTMCs and PAs. For PDTMCs, this is still ongoing research. We present some details later in this thesis.
2.3.1.1 Model checking reachability properties on DTMCs
Assume in the following a DTMC D = (S, sI, P, L) with a single initial state and a set of target
2.3. SPECIFICATIONS FOR PROBABILISTIC MODELS
of finally reaching a state from T when starting at a specific state s ∈ S, we consider all infinite paths in PathsD
inf(s) that contain a state from T . Formally, the set of paths that contribute to the
probability of reaching T from a state s is given by
◊T(s) = {π ∈ PathsDinf(s) | ∃i.target∈ L(π(i))}
where we overload ◊T to both denote a set of paths and a reachability property. We sometimes also omit s, if it is clear from the context, e. g., if s is the initial state of a DTMC.
The set ◊T(s) it measurable, as it corresponds to the union of all cylinder sets of finite paths from PathsD fin(s, T ): ◊T(s) = [ π∈PathsD fin(s,T ) cyl(π)
In order to compute probabilities, we first observe that for π, π0 ∈ PathsD
fin(s, T ) with π ∈
prefD(π0) it holds that cyl(π0) ⊆ cyl(π). However, if π and π0are not prefixes of each other then cyl(π) ∩ cyl(π0) = ;. Thus we can restrict the considered paths to the ones that end when first
visiting a target state:
◊Tfin(s) = {π ∈ PathsDfin(s, T )
∀0 ≤ i < |π|. π(i) /∈ T}
As no path is a prefix of another one in this set, the probability of this set can be computed by the sum of the path probabilities using the measure for finite paths, see Section 2.2.1:
PrD
s (◊T (s)) =
X
π∈◊Tfin(s) P(π)
Having a measurable set, we recall the property at hand: ϕ = P\λ(◊T ) with \ ∈ {<, ≤, >, ≥},
λ∈ [0, 1] ⊆ R and T ⊆ S. We use the notation D |= P≤λ(◊T ) to express that PrDs
I(◊T (sI)) is less than or equal to the bound λ ∈ [0, 1] ⊆ R.
Intuitively, the probability of all paths starting in the initial state sIhas to be inside the interval
defined by \λ. Prior to checking D |= P≤λ(◊T ), the states that cannot reach a target state can be safely removed from the DTMC D.
Definition 23 (Relevant states of a DTMC) Given a DTMC D = (S, sI, P, L) and a set of target states T, a state s ∈ S is called relevant for D and T if
PathsDfin(s, T ) 6= ;. By SD
rel(T) we denote the set of relevant states for D and T. States from S \ Srel(T)D are called
irrelevant for D and T. The set SD
2.3. SPECIFICATIONS FOR PROBABILISTIC MODELS target states, if we assume the state space to be finite.
The property P≤λ(◊T ) for DTMC D is model checked by computing PrDs (◊T ) for all states
s ∈ S and comparing this probability for the initial state with the bound λ. The probabilities ps= PrD
s (◊T ) are obtained as the unique solution of the following linear equation system [BK08,
p. 760]: ps= 1 if s ∈ T 0 if s 6∈ SD rel(T) P s0∈SP(s, s0) · ps0 otherwise.
For some applications it might be necessary to remove all irrelevant states together with their incident transitions. In this case, ps> 0 holds for all remaining states.
2.3.1.2 Model checking reachability properties on PAs
For PAs, computing reachability probabilities gives rise to the question how the nondeterminism should be resolved. For our setting, we require that a reachability property has to hold for all possible schedulers.
Formally, for a formula ϕ = P≤λ(◊T ) and a PA M we have M |= ϕ iff for all schedulers σ∈ SchedM it holds that Mσ|= ϕ, i. e., in the induced DTMC Mσthe property is satisfied.
As a first step for verifying reachability properties on PAs, we identify again the set of relevant states. For the complementary set of irrelevant states the reachability probability is 0.
Definition 24 (Relevant states of PAs) Let M = (S, sI, Act, P , L) be a PA and T ⊆ S a set of target states. Then
SM
rel(T)= {s ∈ S | ∃σ ∈ SchedM. PrMs σ(◊T (s)) > 0}
is the set of relevant states for T. If s 6∈ SM
rel(T) then s is called irrelevant for T.
Intuitively, the irrelevant states for T are the ones for which no scheduler exists such that a state from T is reachable. These states can be computed in linear time by a backward reachability analysis on the PA M with a finite state space [BK08, Algorithm 46]. The removal of irrelevant states does not affect the reachability probabilities. To check whether Mσ|= P≤λ(◊T ) holds for
all schedulers σ of the PA M , it suffices to consider a memoryless deterministic scheduler σ∗
which maximizes the reachability probability for ◊T and check whether PrMσ∗
sI (◊T ) ≤ λ [BK08, Lemma 10.102], i. e., if the probability bound is exceeded in the DTMC that is induced by σ∗.
2.3. SPECIFICATIONS FOR PROBABILISTIC MODELS
ps= PrMs σ∗(◊T ) for each s ∈ S can be characterized by the following equation system:
ps= 1 if s ∈ T 0 if s 6∈ SM rel(T) max P s0∈Sµ(s 0) · p s0| ∃α. (µ, α) ∈ P (s) otherwise.
This equation system can for instance be solved by using value iteration or policy iteration. It can also be transformed into a linear optimization problem that yields the maximal reachability prob- ability together with an optimal scheduler [BK08, Theorem 10.105]. In this case the probability of irrelevant states needs to be explicitly set to 0.