(5.31) Constraint (5.29) states the time available for player B to travel from location Y to location Z is at most the difference in arrival-times of the players. Hence player B may travel no further than
tx - ty from location Y. Constraints (5.30)-(5.31) are the pair constraints applying at time tx .
The set of locations Z satisfying constraints (5.29)-(5.31) is called a feasibility window, through which player B must pass at time tx. If such a location Z exists then the window scenario is called window feasible. If no such location Z exists then player A can safely play the contingent strategy to guarantee either A [> (il -+i2) or A [> (i2-+it} depending upon the
location of player B at time tx .
Lemma 5.2.1
The set of locations Z satisfying constraints (5.29)-(5.31) corresponds to the intersection of the following three circles.
Proof:
Cl : Centre at prize i l and radius r l = dXi2 + di1i2
C2 : Centre at prize i2 and radius r2 = dXi) + di)i2
C3 : Centre at Y and radius r3 = tx - ty
Circle Cl corresponds to constraint (5.31), circle C2 corresponds to constraint (5.30) and
circle C3 corresponds to constraint (5.29). •
Hence the feasibility window may be determined via a geometric construction. Algorithm 5.1 WINDOW FEASIBLE presents an algorithm for determining if the three circles overlap (and hence if
3 a feasibility window) by determining if C3 n CI =F 0, C3 n C2 =F 0 and (C3 n ct } n (C3 n C2 ) =F 0. Example 5.2.2
Consider the three prize problem of Figure 5.7(a). At time ty player B is at location Y = B and player A is at location A. Player A is unconditionally committed to prize 3 and will arrive at the location of prize 3 (location X) at time tx = ty + dA3. Figure 5.7(a) illustrates the
geometric construction of the corresponding feasibility window. Since the three circles have a common intersection then the scenario A [> 3 and B [> {1, 2} is window feasible. Player B must
play so as to reach the feasibility window (the intersection of the three circles) at time tx .
Figure 5.7(b) illustrates a scenario which is not window feasible since the three circles have
no common intersection. 0
5 . 3 Strategies for Two Prize Problem
Suppose that there are two prizes {i1 , i2} and that the prize and pair constraints (5.1)-(5.4) hold with strict inequality as in case (viii) of Section 5.1.3. We now turn to the design of strategies
Algorithm 5.1 procedure WINDOW FEASIBLE
end
Input: Y, i1 , i2 , rI , r2 , r3 '
Output : infeasible or (feasible and t/Jb t/J2 , d1 , d2)'
if ( (r1 �
dyi1)
or (r3 �dYiJ
or (r2 �dYi2)
or (r3 �dYi2))
thenreturn(feasible)
end
/ / Infeasible if C3 n Cl = 0 or C3 n C2 = 0.
if ( (r3 + r 1 <
dYil)
or (r3 + r2 <dYi2))
thenreturn (infeasible)
end
/ / Feasible {::} (C3 n Cd n (C3 n C2) # 0.
t/J1 f- bearing of prize i 1 from location Y
t/J2 f- bearing of prize i2 from location Y
if
(dYil
= r1 + r3) then d1 f- 0 else r� +d�.
-r? x f- 1 12dYi1
Y f- - x2 d1 f- arctan '#.. x end if(dYi2
= r2 + r3 ) then d2 f- 0 else 2 + d2 2 r3Y'
- r2 x f- 122dYi2
Y f- - x2 d2 f- arctan '#.. x endif (d1 + 82 < acute angle between bearings t/J1 and t/J2) then
return( infeasible)
else
return( feasible) end
106 Optimal and Heuristic Analysis of Tiny Problems
3 . . . :\ . . . 'A
(a) Feasibility Construction (b) No Common Intersection
Figure 5.7: Three Prize Problem
for this dynamic case of the Two Prize Problem. Such strategies are STEP-MONITORS since the decision problem is to determine which one-step move to make rather than which prize to target. The case in which dAil � A or dAi2 � A is considered separately in Section 5.3.8; until then, suppose that dAil > A and dAi2 > A. Shortly we will propose a framework for such two prize strategies, but firstly some preliminaries are necessary.
5 . 3 . 1 Slack Variables
Define the following slack variables corresponding to constraints (5.1)-(5.4).
81 = dEil - dAil
82 = dEi2 - dAi2
83 = dAil + dil i2 - dEi2
84 = dAi2
+
dil i2 - dEilHence the two prize case in which we are interested is characterised by 81 > 0, 82 > 0, 83 > 0
and 84 > O. By simple substitutions we obtain the following result.
Result 5.3.1
81 + 82
+
83 + 84 = 2dili281 + 83 = dil '2
+
dEil - dBi281 + 84 = dil i2 + dAi2 - dAil
82 + 83 = dil i2 + dAil - dAi2
82 + 84 = dil i2
+
dBi2 - dEil5.3. 1 . 1 Objective and Constraints
The objective of each player is to maximize the value in prizes claimed. We can now formulate this objective in terms of Assumption 5.1.2 and the slack variables.
Player A must ensure that the prize constraints continue to hold after making a one-step move; hence 81 > 0 and 82 > 0 are constraints. If this is possible, then player A's objective is to achieve either 83 ::; 0 or 84 ::; O. If this is not possible then A-() argmax{ Vii ' Vi2 } is critical.
Player B must ensure that the pair constraints continue to hold after making a one-step move; hence 83 > 0 and 84 > 0 are constraints. If this is possible then player B's objective is to achieve either 81 ::; 0 or 82 ::; O. If this is not possible then B-() argmin {Vii , Vi2 } is critical.
Result 5.3.1 has several consequences in terms of the tradeoff between both players attempting to satisfy their objective. Since 81 + 82 + S3 + S4 = 2dili2 is a constant, minimizing 83 + 84 is equivalent to maximizing 81 + 82 and vice versa. In this way the players' objectives are opposed. However, if 83 « 84 then player B can "defend" 83 via B-()i2 and similarly if 83 » 84. Suppose player A converges to prize il and the prize and pair constraints continue to hold. Then dAi2 - dAil
converges to di1 i2 ' 82 + 83 converges to 0 and hence 82 and 83 both converge to O. Thus as player A anticipates achieving its objective, player B also anticipates achieving its objective.
5.3.1.2 Critical Values of Slack Variables
A one-step move is safe for player A if it ensures that both prize constraints continue to hold. A one-step move is safe for player B if it ensures that both pair constraints continue to hold. We now give some computable bounds on the slack variables such that 3 a safe one-step move for each player.
Critical values of 81 and 82. Let (}A be the acute angle between the prizes from the
perspective of player A. The maximum decrease in 81 from one step occurs when A-oi2 and B-()il and hence
(5.32)
by the Cosine Law. Similarly the maximum decrease in 82 from one step occurs when A-oil and B-()i2 and hence
(5.33) If 81 > 8rit or 82 > 82rit then one of A-()il or A-()i2 is safe, i.e., both prize constraints will continue to hold after the one-step moves have been made. If 81 ::; 8�rit and 82 :::; 82rit then player A is potentially in danger of relinquishing the claim to one of the prizes as neither A-()i1 nor A -()i2 is safe.
Critical values of 83 and 84 . Let (}B be the acute angle between the prizes from the
perspective of player B. The maximum decrease in S3 from one step occurs when both A-()il and B-()il and hence
108 Optimal and Heuristic Analysis of Tiny Problems
by the Cosine Law. Similarly the maximum decrease in 84 from one step occurs when A-oi2 and B-oi2 and hence
(5.35) If 83 > 83rit or 84 > 8�rit then one of B-oil or B-oi2 is safe, i.e., both pair constraints will continue to hold after the one-step moves have been made. If 83 :::; 8rit and 84 :::; 8rit then player B is potentially in danger of relinquishing the claim to one of the sequences A I> (il -+i2)
or A I> (i2-+il) as neither B-oil nor B-oi2 is safe. 5.3.2 Safety Windows
When one of the 8i values falls below its critical value we must determine which one-step moves are safe. The set of safe one-step moves correspond to a window through, which the player must play which we call a safety window.
5.3.2.1 Player A Safety Window
At some time to , suppose 81 :::; 8�rit or 82 :::; 8rit, but dAil > tl and dAi2 > tl.
We wish to determine a location, Z, for player A at time to + tl such that 81 � 0 and 82 � 0
definitely hold at time to + tl. Then such a location Z must satisfy constraints (5.36)-(5.38). tl > dAZ dEil -tl > dZil dEi2 -tl > dZi2 (5.36) (5.37) (5.38)
This is a window feasibility subproblem as in Section 5.2 except that it is player A who wishes to determine a feasibility window and player B has no committed target location at time to + tl.
Hence apply Algorithm 5.1 WINDOW FEASIBLE to determine if such a location Z exists. The parameters required are the prizes il and i2, location Y is the location of player A and
Tl = dBil -tl
T2 = dEi2 -tl T3 = tl
If such a location Z exists then this game position is A-safe and the corresponding feasibility window is called the A-safety-window.
5.3.2.2 Player B Safety Window
At some time to, suppose 83 :::; 83rit or 84 :::; 8�rit , but dAil > tl and dAi2 > tl.
We wish to determine a location, Z, for player B at time to + tl such that 83 � 0 and 84 � 0
definitely hold at time to + tl. Then such a location Z must satisfy constraints (5.39)-(5.41).
tl > dBZ
(dAil -tl) + dili2 > dZi2
(dAi2 - tl) + dil i2 > dZil
(5.39) (5.40) (5.41)
This is a window feasibility su.bproblem as in Section 5.2 except that player A has no committed target location at time to + �.
Hence apply Algorithm 5.1 WINDOW FEASIBLE to determine if such a location Z exists. The parameters required are the prizes il and i2 , location Y is the location of player B and
Tl = dAi2 -� + di1i2 T2 = dAi2 -� + di1i2 T3 �
If such a location Z exists then this game position is B-safe and the corresponding feasibility window is called the B-safety-window.
5.3.3 Two Prize Strategy Framework
We now propose a framework for a two prize strategy consisting of two components.
(a). Construction of a target window (an interval of bearings) through which the player must play its one-step move. The defau.lt target window (that which is least restrictive) is the window which spans the two prizes via the acute angle between their bearings.
(b). Selection of a target location within the target window which the player will move towards as its one-step move.
5.3.3.1 Construction of Target Window
Suppose either 81 � srit or 82 � S2rit. If the game position is A-safe, then the A-target-window is the A-safety-window; otherwise the A-target-window is the default window.
Suppose both 81 > srit and 82 > srit . Then the A-target-window is the default window.
Suppose either 83 � srit or 84 � s�rit. If the game position is B-safe, then the B-target-window is the Bsafety-window; otherwise the B-target-window is the default window.
Suppose both 83 > 83rit and S4 > 8rit. If B [> {il , i2} is median-feasible, then the B-target
window is the B-median-window of Section 5.1.4; otherwise the B-target-window is the default window.
Note. IT player B plays to the B-median-window then B [> {il ' i2} remains median-feasible and
the B-median-window will continue to enlarge until eventually SI � 0 or 82 � 0 is achieved. However, it is not necessarily always true that if a player plays to a safety-window then the resulting game position will also have a safety-window.
5.3.3.2 Selection of Target Location
Sections 5.3.4-5.3.7 present simple heuristic strategies for the selection of a either a target-prize or a target-bearing. If a target-prize is selected then it is translated to the corresponding end of the target-window. IT a target-bearing is selected and this is outside the target-window then this also is translated to the corresponding end of the target-window.
110 Optimal and Heuristic Analysis of Tiny Problems
Algorithm 5.2 strategy TWO-PRIZE AHEAD PURE-ATTACK
end
k' t- argminkE{ I ,2}5k+2
target t-iko
Algorithm 5.3 strategy TWO-PRIZE AHEAD PURE-DEFEND
end
k* t-argminkE{ I ,2}5k
target t-iko
5 .3.4 A Family of ATTACK and DEFEND Strategies
The basic strategy requirement for player A is to ensure 51 > 0 and 52 > 0 continue to hold and attempt to force either 83 � 0 or 54 � O. This gives rise two strategies for player A: Algorithm 5.2
TWO-PRIZE AHEAD PURE-ATTACK and Algorithm 5.3 TWO-PRIZE AHEAD PURE-DEFEND.
The basic strategy requirement for player B is to ensure 83 > 0 and 54 > 0 continue to hold and attempt to force either 81 � 0 or S2 � O. This gives rise to two strategies for player B:
Algorithm 5.4 TWO-PRIZE BEHIND PURE-ATTACK and Algorithm 5.5 TWO-PRIZE BEHIND PURE
DEFEND.
The pure attack strategies persistently attack the weakest of two constraints and the pure de fend strategies continually defend the weakest of two constraints. Rather than attack the weakest of two constraints, an alternative pure attack strategy is to attack the strongest of two constraints, thus continually reducing a ceiling on the opponent's overall strength, as in Algorithm 5.6 TWO
PRIZE AHEAD CEILING-ATTACK and Algorithm 5.7 TWO-PRIZE BEHIND CEILING-ATTACK.
Consider the following scenarios.
(i). SI small, S2 small, S3 large, S4 large. Player A should defend and player B should attack. (ii). SI large, S2 large, S3 small, S4 small. Player A should attack and player B should defend. (Hi). SI large, S2 small, S3 large, S4 small. Player A-<>i2 is the obvious attacking or defensive
move. Defence implies B-<>il and attack implies B-<>i2.
Algorithm 5.4 strategy TWO-PRIZE BEHIND PURE-ATTACK
end
k* t-argminkE{I,2}Sk target +-iko
Algorithm 5 . 5 strategy TWO-PRIZE BEHIND PURE-DEFEND
end
k* t- argminkE{1,2}5k+2
target t- ik.
Algorithm 5.6 strategy TWO-PRIZE AHEAD CEILING-ATTACK
end
k* t- argmaxkE{I,2}Sk+2
target t-ik.
(iv). SI large, S2 small, S3 small, S4 large. Player B-<Ji2 is the obvious attacking or defensive move. Defence implies A -<Ji2 and attack implies A -<Jil .
In summary, player A should defend if and only if min {SI , S2} is small and player B should defend if and only if min{s3, s4 } is small. Algorithm 5.8 TWO-PRIZE AHEAD THRESH-DEFEND and Algorithm 5.9 TWO-PRIZE BEHIND THRESH-DEFEND present two resulting strategies for a threshold parameter Sthresh .
5.3.5 A Family of TIT FOR TAT Strategies
Result 5.3.1 shows that SI + S3 decreases as player B gets closer to prize il and S2 + S3 decreases as player A gets closer to prize il .
Consider what conditions would imply that S3 consistently decreases in a prolonged play of a two prize problem instance. Now B-<Ji2 implies that S3 cannot decrease, so a necessary condition is that B-<Jil . Also B-<Jil implies that SI + S3 consistently decreases at a rate depending only on player B's location. Eventually player A must approach prize il to claim the sequence il -ti2 and A-<Jil implies that SI cannot decrease. Hence it is reasonable to expect that A-<Jil and B-<Jil consistently for 53 to monotonically decrease ..
Even the above is by no means certain to guarantee that S3 approaches zero �it is possible that S2 > 0 may not hold throughout. However, it is enough to suggest a family of two prize strategies for both players based on the following two reasonable expectations for prolonged play:
Algorithm 5.7 strategy TWO-PRIZE BEHIND CEILING-ATTACK
end
k* t- argmaxkE{l,2}Sk
112 Optimal and Heuristic Analysis of Tiny Problems
Algorithm 5.8 strategy TWO-PRIZE AHEAD THRESH-DEFEND
end
Input: Sthresh / / Threshold parameter. if (min{ SI , S2} � Sthresh) then
/ / A-attack. k* t- argminkE{I .2} Sk+2 else / / A-defend. k* t- argminkE{I.2} Sk end target t-ik.
Algorithm 5.9 strategy TWO-PRIZE BEHIND THRESH-DEFEND
end
Input: Sthre8h / / Threshold parameter.
if (min{ S3, S4} � Sthresh) then
/ / B-attack. k* t-argminkE{I.2}Sk else / / B-defend. k* t-argminkE{1.2}Sk+2 end target t-ik.
Algorithm 5 . 1 0 strategy TIT FOR TAT BEHIND (BEARING)
end
/ / Target opposite to opponent 's previous bearing.
target t- Bearing
6 . OB from 1/1Bi2 towards 1/1Bil .
(i
)
. Player A would ideally like to consistently target one prize, say prize it , and discover that player B has also targeted prize it over an reasonable length of time.(ii
)
. On the other hand, if player A targets one prize, say prize i l , then player B would like to ensure that it defends the corresponding slack variable, S3 , by targeting prize i2 ·Each player does not know what its opponent's one-step move will be, but can use observation of the previous move in the history of the game as a substitute.
5.3.5 . 1 TIT FOR TAT BEHIND Strategy
The idea is that player B can avoid the scenario above by targeting the opposite prize to that which player A targeted at the previous iteration. This is an example of a strategy which Axelrod [7] calls TIT FOR TAT in the context of the iterated Prisoner's Dilemma game of Section 2.4.5 .
If the players were restricted to travelling only towards prizes then this would be straightfor ward. As no such restriction applies, we need a definition of an opposite move to the opponent's previous move. Such a definition must retain the criterion that if player A targets a prize then player B should target the opposite prize.
The idea is to determine what "weighting" of the prizes player A targeted at the previous iteration and then apply the weighting in reverse. Two possibilities include:
B earings. Let 1/1Ail be the bearing of prize it from player A's immediately preceding location and let 1/1Ai2 be the bearing of prize i2 from player A's immediately preceding location. Let 1/1 A- be the bearing that player A moved at the immediately preceding step. IT 1/1 A
is outside the acute angle between 1/1 Ail and 1/1 Ai2 ' then redefine 1/1 A- as 1/1 Ail or 1/1 Ai2 '
whichever is the closest bearing to the existing 1/1 A -. Let 0 Ail be the acute angle between
1/1Ail and 1/1A- and let OAi2 be the acute angle between 1/1Ai2 and 1/1A- . Let 'Y = 6 . •
A.}
Let 1/1BiI be the bearing of prize il from player B's current location and let 1/1Bi2 be the bearing of prize i2 from player B's current location. Let OB be the acute angle between
1/1 Bil and 1/1 Bi2 . Let 1/1 Bo be the bearing an angle 'YOB from bearing 1/1 Bi2 towards 1/1 Bil . Player B then targets bearing 1/1 Bo .
The result is Algorithm 5.10 TIT FOR TAT BEHIND (BEARING).
Location. Let ZA- be the displacement from prize it towards prize i2 along the "median line" through the two prizes which player A targeted at the immediately preceding step. IT
Z A - < 0 then set Z A - = O. If Z � diti2 then set Z A - = dil i2 . Player B then targets the
1 1 4 Optimal and Heuristic Analysis o f Tiny Problems
Algorithm 5.11 strategy TIT FOR TAT BEHIND (LOCATION)