Algorithms for Solving Parity Games
3.4 Related work
This survey of algorithms for solving parity games is not exhaustive. In particular, an important class of local search algorithms has not been included here. This class includes strategy improvement algorithms, inspired by policy iteration algorithms for Markov decision processes (Howard [1960], Puterman [2005]) and stochastic games (Filar and Vrieze [1997]), and various pivoting algorithms adapted from the theory of the linear complementarity problem (Cottle et al. [2009]).
The initial impulse for exploring applications of such local search algorithms to parity games has been the observation by Puri [1995] and Stirling [1995]
that there is a polynomial time reduction from parity games to mean-payoff games and to simple stochastic games. The reduction has also been exploited by Jurdzi´nski [1998] to show that the problems of deciding the winner in parity, mean-payoff, discounted, and simple stochastic games are in UP and in co-UP.
V¨oge and Jurdzi´nski [2000] have been inspired by the reduction from parity games to discounted games to devise a discrete strategy improvement algorithm for parity games. This algorithm has been conjectured to have worst-case polynomial running time, but in a recent breakthrough Friedmann [2009] has dashed those hopes by exhibiting a family of parity games with O(n) vertices on which the algorithm performs Ω(2n) iterations. Intriguingly, working with parity games and the discrete strategy improvement algorithm has enabled Friedmann to devise examples of mean-payoff, discounted, and simple stochastic games that make the strategy improvement algorithm
96 Marcin Jurdzi´nski
take exponentially many steps, a feat which had remained elusive ever since Howard [1960] proposed policy iteration algorithms for Markov decision processes. Fearnley [2010b] has adapted Friedmann’s examples to make Howard’s policy iteration take exponentially many iterations also on Markov decision processes, and hence exhibiting a surprising weakness of policy iteration and strategy improvement algorithms, even on (stochastic) one-player games. On the other hand, the proposal of Fearnley [2010a] to consider non-oblivious strategy improvement algorithms is a promising new way to explore strategy improvement in the quest for polynomial time algorithms, despite the damage inflicted to this line of research by Friedmann’s examples.
Randomised variations of the strategy improvement technique, sometimes referred to as the Random Facet algorithms, have been proposed for simple stochastic games by Ludwig [1995] and Bj¨orklund and Vorobyov [2007].
They were inspired by a subexponential randomised simplex algorithm of Matouˇsek et al. [1996], and they were the first subexponential (randomised) algorithms for solving parity, mean-payoff, discounted, and simple stochastic games. A recent result of Friedmann et al. [2010] establishes that the Random Facet algorithm requires super-polynomial time on parity games.
An important corollary of the applicability of strategy improvement algo-rithms to solving games on graphs is that the search problems of com-puting optimal strategies in parity, mean-payoff, discounted, and simple stochastic games are in PLS (Johnson et al. [1988]). Moreover, those search problems are also known to be in PPAD (Papadimitriou [1994]) because they can be reduced in polynomial time (G¨artner and R¨ust [2005], Jurdzi´nski and Savani [2008]) to the P-matrix linear complementarity problem (Cottle et al. [2009]). The latter is in PPAD since it is processed by Lemke’s algorithm (Lemke [1965], Papadimitriou [1994], Cottle et al. [2009]). It follows that the problems of computing optimal strategies in games on graphs are unlikely to be complete for either of the two important complexity classes of search problems, unless one of them is included in the other.
The reductions from games on graphs to the P-matrix linear complemen-tarity problem have recently facilitated applications of classical algorithms for the linear complementarity problem to solving games. Fearnley et al. [2010]
and Jurdzi´nski and Savani [2008] have considered Lemke’s algorithm , the Cottle–Danzig algorithm , and Murty’s algorithm for discounted games, and they have shown that each of those pivoting algorithms may require exponential number of iterations on discounted games. Randomised versions of these algorithms require further study.
Algorithms for Solving Parity Games 97
References
H. Bj¨orklund and S. Vorobyov. A combinatorial strongly subexponential strategy improvement algorithm for mean payoff games. Discrete Applied Mathematics, 155(2):210–229, 2007.
A. Browne, E. M. Clarke, S. Jha, D. E. Long, and W. R. Marrero. An improved algorithm for the evaluation of fixpoint expressions. Theor. Comput. Sci., 178 (1–2):237–255, 1997.
R. W. Cottle, J.-S. Pang, and R. E. Stone. The Linear Complementarity Problem, volume 60 of Classics in Applied Mathematics. Society for Industrial & Applied Mathematics, 2009.
E. A. Emerson and C. Jutla. Tree automata, μ-calculus and determinacy. In Foundations of Computer Science (FOCS), pages 368–377. IEEE Computer Society Press, 1991.
E. A. Emerson and C.-L. Lei. Efficient model checking in fragments of the proposi-tional mu-calculus (Extended abstract). In Logic in Computer Science (LICS), pages 267–278. IEEE Computer Society Press, 1986.
E. A. Emerson, C. S. Jutla, and A. P. Sistla. On model-checking for fragments of μ-calculus. In Computer-Aided Verification (CAV), volume 697 of LNCS, pages 385–396. Springer, 1993.
J. Fearnley. Non-oblivious strategy improvement. In Logic for Programming, Artificial Intelligence, and Programming (LPAR), 2010a. To appear.
J. Fearnley. Exponential lower bounds for policy iteration. In International Collo-quium on Automata, Languages and Programming (ICALP), volume 6199 of LNCS, pages 551–562. Springer, 2010b.
J. Fearnley, M. Jurdzi´nski, and R. Savani. Linear complementarity algorithms for infinite games. In Current Trends in Theory and Practice of Computer Science (SOFSEM), volume 5901 of Lecture Notes in Computer Science, pages 382–393.
Springer, 2010.
J. Filar and K. Vrieze. Competitive Markov Decision Processes. Springer, Berlin, 1997.
O. Friedmann. An exponential lower bound for the parity game strategy improvement algorithm as we know it. In Logic in Computer Science (LICS), pages 145–156.
IEEE Computer Society Press, 2009.
O. Friedmann, T. D. Hansen, and U. Zwick. A subexponential lower bound for the Random Facet algorithm for parity games. Manuscript, April 2010.
B. G¨artner and L. R¨ust. Simple stochastic games and P-matrix generalized linear complementarity problems. In Fundamentals of Computation Theory (FCT), volume 3623 of Lecture Notes in Computer Science, pages 209–220. Springer, 2005.
R. A. Howard. Dynamic Programming and Markov Process. MIT Press, Cambridge, Massachusetts, 1960.
D. S. Johnson, C. H. Papadimitriou, and M. Yannakakis. How easy is local search?
J. Comput. Syst. Sci., 37(1):79–100, 1988.
M. Jurdzi´nski. Small progress measures for solving parity games. In Symposium on Theoretical Aspects of Computer Science (STACS), volume 1770 of Lecture Notes in Computer Science, pages 358–369. Springer, 2000.
M. Jurdzi´nski. Deciding the winner in parity games is in UP∩ co-UP. Inf. Process.
Lett., 63(3):119–124, 1998.
98 Marcin Jurdzi´nski
M. Jurdzi´nski and R. Savani. A simple P-matrix linear complementarity problem for discounted games. In Computability in Europe (CiE), volume 5028 of LNCS, pages 283–293. Springer, 2008.
M. Jurdzi´nski, M. Paterson, and U. Zwick. A deterministic subexponential algorithm for solving parity games. SIAM J. Comput., 38(4):1519–1532, 2008.
C. E. Lemke. Bimatrix equilibrium points and mathematical programming. Man-agement Science, 11:681–689, 1965.
W. Ludwig. A subexponential randomized algorithm for the simple stochastic game problem. Inf. Comput., 117(1):151–155, 1995.
D. A. Martin. The determinacy of Blackwell games. J. Symb. Log., 63(4):1565–1581, 1998.
J. Matouˇsek, M. Sharir, and E. Welzl. A subexponential bound for linear program-ming. Algorithmica, 16:498–516, 1996.
R. McNaughton. Infinite games played on finite graphs. Ann. Pure Appl. Logic, 65 (2):149–184, 1993.
C. H. Papadimitriou. On the complexity of the parity argument and other inefficient proofs of existence. J. Comput. Syst. Sci., 48:498–532, 1994.
A. Puri. Theory of Hybrid Systems and Discrete Event Systems. PhD thesis, University of California, Berkeley, 1995.
M. L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Pro-gramming. Wiley Blackwell, 2005.
S. Schewe. Solving parity games in big steps. In FSTTCS 2007, volume 4855 of Lecture Notes in Computer Science, pages 449–460. Springer, 2007.
H. Seidl. Fast and simple nested fixpoints. Inf. Process. Lett., 59(6):303–308, 1996.
C. Stirling. Local model checking games (Extended abstract). In Concurrency Theory (CONCUR), volume 962 of Lecture Notes in Computer Science, pages 1–11. Springer, 1995.
J. V¨oge and M. Jurdzi´nski. A discrete strategy improvement algorithm for solving parity games (Extended abstract). In Computer Aided Verification (CAV), volume 1855 of Lecture Notes in Computer Science, pages 202–215. Springer, 2000.
W. Zielonka. Infinite games on finitely coloured graphs with applications to automata on infinite trees. Theor. Comput. Sci., 200(1–2):135–183, 1998.