Solving Reachability-Price Minimisation Problem

2.2.1.1. Optimality Equations

LetG= (V,E,F,π)be a finite graph with final vertices. To keep the discussion simple, we assume that the graphGsatisfy the following assumption:

ASSUMPTION 2.2.2. For every vertex v ∈ V we have that minimum reachability-price is finite.

Before we consider the problem in full generality, we try to solve a simpler problem and assume the following:

ASSUMPTION2.2.3. The graphGdoes not have any zero cycles.

For the finite price graphG= (V,E,F,π)let us consider the following set of equations:

P(v) = ( 0, ifv_∈F, min(v,w)∈E π(v,w) +_P(w) , otherwise. We denote these equations byOERP

Min(G). We say that a functionp:V→Ris a solution

of the optimality equations _OERP_Min(G), and we write p _|= _OERP

Min(G), if all equations in

OERPMin(G)hold for the valuationsP(v)7→ p(v).

The following proposition states the relation between optimality equationsOERP

Min(G)

and a solution of minimum reachability-price problem.

PROPOSITION 2.2.4(A solution of optimality equations gives optimal reachability-price). Let G be a finite priced graph. Under Assumptions 2.2.2 and 2.2.3 we have that p _|= OERPMin(G)impliesRP∗(v) = p(v)for allv∈V.

PROOF. We prove the lemma in two parts. First we show that for every strategyµ∈Σ, we

havep(v)_≤RP(Run(v,µ))for everyv_∈V. In the second part, we show that there exists a positional strategyµ0 ∈Πsuch that for every vertexv∈Vwe haveRP(Run(v,µ0)) = p(v).

– Let Run(v,µ) = _hv = v0,v1, . . .i be the run from the vertex v according to the

strategyµ. Sincep|=OERP_Min(G), for alli≥0, we have the following. p(v_i) = 0 ifv_{i ∈} F

p(vi) ≤ π(vi,vi+1) +p(vi+1), otherwise . (2.2.1)

Under Assumptions 2.2.2 and 2.2.3 we have that there is an indexn _∈ Nsuch that Stop(Run(v,µ)) = n. Summing (2.2.1) side-wise for 0 _≤ i < n, we get the following. p(v) ≤ n

∑

i=1 π(vi,vi−1) +p(vn).

Sincevn∈ F, we havep(vn) =0 and we get the desired inequality.

p(v) _≤ RP(Run(v,µ)).

– For a set W the function Choose : 2W → W is defined such that for every set

X _⊆ W we have Choose(X) ∈ X. In other words, Choose is a choice function which for every non-empty set selects an element of this set. We fix an arbitrary choice function for technical reasons, so that we have a way to canonically choose an arbitrary element from every non-empty set.

Letµ0 ∈Πbe the following positional strategy: µ0(v) =Choose(argmin

w∈V

π(v,w) +p(w) : (v,w)∈ E ).

Sincep_|= _OERP_Min(G), for such strategyµ0it is straightforward to notice that for

allv_∈ V_\Fwe have:

p(v) = π(v,µ0(v)) +p(µ0(v)). (2.2.2)

Let the run Run(v,µ0)behv = v0,v1,v2, . . .i. Again under Assumptions 2.2.2

and 2.2.3 we have that there is an index n ∈ Nsuch that Stop(Run(v,µ0)) = n.

Summing (2.2.2) for alli, 0≤i<n, we get the following.

p(v) =

n

∑

i=1

π(vi,vi−1) +p(vn).

Sincevn∈ F, we havep(vn) =0 and we get the desired equality.

p(v) = RP(Run(v,µ0)).

We have thus shown that: for every strategyµ ∈ Σ, we have p(v) ≤ RP(Run(v,µ))for

everyv_∈ V; and there exists a positional strategyµ0 ∈ Πsuch thatRP(Run(v,µ0)) = p(v)

for every vertexv∈V. Hence it follows thatp(v) =RP_∗(v)for every vertexv∈V.

From the second part of the proof of the previous proposition, the following proposition follows:

PROPOSITION2.2.5(A solution of optimality equations gives positional optimal strategy). LetGbe a priced finite graph. Under Assumptions 2.2.2 and 2.2.3 we have that if there exists a solution of optimality equations_OERP_Min(G)then there exists a positional optimal strategy for minimum reachability-price problem.

The following example shows that a solution of the optimality equations OERP

Min(G)

does not give the correct reachability-price, if the graphGcontains a zero cycle.

EXAMPLE2.2.6(Problems with zero cycles). Consider the graph shown in Figure 2.1. Notice that the graph contains zero cycles. For example the price of the cycle _hv1,v3,v1i is 0.

Optimality equations_OERP_Min(G)are as follows: P(v2) = 0 P(v1) = min 20+_P(v2),P(v3) P(v3) = min P(v1) .

In this case infinitely many solutions of optimality equations exist; as all the functions

p : V _→ R such that p(v1) = k, p(v2) = 0 and p(v3) = k, for any k ≤ 20, satisfy the

optimality equations. However only one (i.e., whenk =20) of these functions gives correct minimum reachability-price.

To solve the reachability-price problem for general graphs, we propose the following optimality equations: (P(v),D(v)) = ( (0, 0), ifv_∈ F, minlex (v,w)_∈E (π(v,w) +_P(w), 1+_D(w)) , otherwise. For the sake of notational convenience we reuse the name OERP

Min(G) for these set of

equations as well. We say that the functionsp:V_→Randd:V_→Nsatisfy the optimality equations_OERP_Min(G)and we write(p,d) _|= _OERP

Min(G), if all equations in OERPMin(G)hold

for the valuations_P(v)_7→ p(v)and_D(v)_7→d(v).

The introduction of _D variables solves the problem due to zero cycles. The proof of the following proposition is similar to the proof of Proposition 2.2.4 and hence we skip the proof here. We provide a complete proof of a lemma similar to this proposition, but in the context of timed automata (see Proposition 5.1.3).

PROPOSITION 2.2.7(A solution of optimality equations gives optimal reachability-price). Under the Assumption 2.2.2 we have that if (p,d) |= OERP

Min(G) then we have p(v) = RP_∗(v)for allv_∈ V.

PROPOSITION2.2.8(A solution of optimality equations gives positional optimal strategy). Under the Assumption 2.2.2 we have that if there exists a solution(p,d)of the optimality equations _OERP_Min(G) then there exists a positional optimal strategy for reachability-price problem.

In the rest of this subsection, we give a constructive proof of the following proposition by giving a strategy improvement algorithm which yields a solution of the optimality equations_OERP_Min(G).

PROPOSITION2.2.9(Existence of a solution). If the Assumption 2.2.2 holds then there exists a solution of the optimality equationsOERP

Min(G).

2.2.1.2. Solving0-player Optimality Equations

For a graphG = (V,E,F)and a positional strategyµ ∈ ΠMin, we define the graphGµ = (V,E0,F)such thatE0 = _{(v,µ(v)) : v_∈V_}.

If the graphG= (V,E,F)is such that every vertex has a unique successor, i.e., for every

v_∈Vwe have that the setE(v) =_{w : (v,w)_∈ E_}is a singleton, then we call such a graph a 0-player graph. Observe that for an arbitrary graphGand a positional strategyµ∈ ΠMin

the graphGµis a 0-player graph. A 0-player priced graph is defined in straightforward

manner.

For a 0-player priced graph G = (V,E,F,π), the optimality equations OERP

Min(G)can

be rewritten in the following manner:

(P(v),D(v)) =

(

(0, 0), ifv_∈F,

(π(v,E(v)) +_P(E(v)), 1+_D(E(v))), otherwise.

Remark. For brevity we have abused the notation to denote the unique vertexwsuch that

(v,w)∈ Eby the singleton setE(v)containingw.

For a 0-player priced graphG, we write_OERP(G)for these set of equations. The optimality equations for 0-player reachability problem can be solved in timeO(|V|).

2.2.1.3. Solving1-player Optimality Equations

For given functionsp :V →Randd :V →N, for every vertexv ∈V, let us define the set

M_∗(v,p,d)as follows:

M_∗(v,p,d) =argminlex

w∈V {

(π(v,w) +p(w), 1+d(w)) : (v,w)_∈ E_}.

Consider the strategy improvement functionImprove_Min : Π_×[V _→R]_×[V_→N] _→ Π

defined by:

Improve_Min(σ,p,d)(v) =

(

σ(v) ifσ(v)∈ M_∗(v,p,d)

Choose(M_∗(v,p,d)) otherwise,

wherep : V → R,d :V → N, andv ∈ V. From the definition of the function Improve_Min, the following proposition is immediate.

PROPOSITION2.2.10(Fixed point of Improve_Minare solutions of_OERP_Min(G)). Letµ∈ΠMin

and let(pµ_,dµ_)|=OERP_{(Gµ). If Improve}

Min(µ,pµ,dµ) =µthen(pµ,dµ)|=OE

RP

Input: A graphG= (V,E,F,π) Output: A solution ofOERP

Min(G)

begin 1

(Initialisation). Choose an arbitrary positional strategyµ0∈ ΠMin;

2

Seti=0; 3

repeat 4

(Value Computation). Compute the solution(pi,di)ofOERP(Gµi);

5

(Strategy Improvement). Computeµi+1 =ImproveMin(µi,pi,di);

6 Seti:=i+1; 7 untilµ_i+1≡µi; 8 return(pi,di); 9 end 10

FIGURE2.2. Strategy improvement algorithm to solveOERP

Min(G).

A pseudocode of the strategy improvement algorithm to solve the optimality equations OERPMin(G) is shown as Algorithm 2.2. Before we prove the correctness and termination

properties of the algorithm, we would like to introduce a few notations and their properties. We say that functions p:V →Randd :V→Nsatisfy the set of optimality equations OERP ≥ (G), and we write(p,d)|= OERP≥ (G), if (p(v),d(v)) ≥lex ( (0, 0), ifv_∈ F, maxlex (π(v,w) +p(w), 1+d(w)) : (v,w)_∈ E , otherwise. The following proposition summarise a useful relation between a solution ofOERP

Min(G)

and_OERP_≥ (G).

PROPOSITION 2.2.11. Let p,p_≥ : V → R and d,d_≥ : V → N be such that (p,d) |=

OERP

Min(G)and(p≥,d≥) |= OERP≥ (G). Then we have(p≥,d≥)≥lex(p,d), and if(p≥,d≥)6|= OERP

Min(G)then(p≥,d≥)>lex(p,d).

PROOF. First we establish by induction on d(v)that (p_≥,d_≥)≥lex_{(p,d). The base case,} d(v) = 0, is trivial because, sincev _∈ F, we have that(p(v),d(v)) = (0, 0)and(p_≥,d_≥) ≥ (0, 0).

Letv _∈ V_\Fbe such thatd(v) = n+1 and let(v,w) _∈ Ebe such that(p(v),d(v)) = (π(v,w) +p(w), 1+d(w)). Notice that it implies that d(w) = n and hence by induction hypothesis we have(p_≥(w),d_≥(w))_≥lex_{(p(w),d(w)). Since(p}

≥,d≥)|= OERP≥ (G), we have

(p_≥(v),d_≥(v)) ≥lex _{(π(v,w) +p}

≥(w), 1+d≥(w)) (2.2.3) ≥lex _{(π(v,w) +p(w), 1+d(w))}

The second inequality follows from(p_≥(w),d_≥(w))_≥lex_{(p(w),d(w)). This concludes the}

proof that(p_≥,d_≥)_≥lex_(p,d).

Now we prove that if (p_≥,d_≥) _6|= _OERP_Min(G) then there is a vertex v _∈ V such that

(p_≥(v),d_≥(v))>lex_{(p(v),d(v)). Indeed, if (p}

≥,d≥) 6|= OERPMin(G)then either we have that (p_≥(v),d_≥(v))<lex_{(0, 0)for some vertexv∈ F, or there is some vertexv ∈ V\Fsuch that}

the inequality (2.2.3) is strict and hence we get(p_≥(v),d_≥(v))>lex_(p(v),d(v)).

The following lemma states that every step of strategy improvement algorithm im- proves the strategy.

LEMMA 2.2.12 (Strict strategy improvement). Let µ,µ0 ∈ ΠMin, let (p,d) |= OERP(Gµ)

and(p0,d0)_|= _OERP_(G

µ0), and letµ0 = Improve_Min(µ,p,d). Then(p,d)≥lex(p0,d0)and if µ6=µ0then(p,d)>lex(p0,d0).

PROOF. First we argue that (p,d) |= OERP

≥ (Gµ0), which by Proposition 2.2.11 implies

that(p,d)_≤lex_(p0_,d0_{). Indeed, for everyv∈ V\Fifµ(v) =wandµ}0_{(v) =w}0_{then we have}

(p(v),d(v)) = (π(v,w) +p(w), 1+d(w)), as(p,d)|=OERP₍

Gµ) ≥lex _π(v,w0_{) +p(w}0_{), 1+d(w}0₎

, by definition of Improve_Min. Moreover, if µ 6= µ0 then there is a v ∈ V\F for which the above inequality is strict.

Then(p,d)_6|=_OERP_Min(Gµ0), because every vertex inGµ0has a unique successor, and hence

by Proposition 2.2.11 we conclude that(p,d)>lex_(p0_,d0_).

LEMMA2.2.13(Correctness and termination of the strategy improvement algorithm). The strategy improvement algorithm for OERP

Min(G) terminates in finitely many steps and

returns a solution(p,d)of_OERP_Min(G).

PROOF. Since the total number of positional strategies in a finite priced graph is finite

(bounded from above by |V_||V|_{), this lemma follows directly from Lemma 2.2.12 and} Proposition 2.2.10.

In document Competitive optimisation on timed automata (Page 33-38)