Conclusions for Sections 9 and 10

We have analyzed the black-box complexities of the two combinatorial problems of finding a minimum spanning tree and finding single-source shortest paths trees.

In the MST problem, we could apply many of the techniques developed in previous parts of this thesis. We have shown that access to variation operators of arity greater than one provably helps to decrease runtimes. This raises the question whether there exist “natural” bio-inspired algorithms, which outperform standard mutation-based search heuristics, by using higher arity variation operators. We believe that our work indicates ways to design such algorithms.

For the SSSP problem, the main challenge is in finding reasonable ways to gener- alize the unbiased black-box model by Lehre and Witt to more general search spaces than the hypercube {0, 1}n. We have analyzed three different approaches. Two seem- ingly natural models proved not very useful. In fact, both the generalized unbiased black-box model by Rowe and Vose as well as the structure preserving model intro- duced here in this work turned out to be almost the same as the unrestricted model. These results indicate that much care has to be taken in order to get a reasonable un- biasedness definition for the problem at hand. For the SSSP problem, the redirecting unbiased black-box model seems most promising.

For both the MST and the SSSP problem, the differences between the ranking- based model and their basic counterparts are negligible. This is due to the nature of the two problems. It would certainly be interesting to investigate other combinatorial problems in the different black-box settings.

Bibliography

[Aar04] Scott Aaronson. Lower bounds for local search by quantum arguments. In Proceedings of the 36th Annual ACM Symposium on Theory of Computing (STOC’04), pages 465–474. ACM, 2004.

[AD11] Anne Auger and Benjamin Doerr. Theory of Randomized Search Heuristics. World Scientific, 2011.

[Ald83] David Aldous. Minimization algorithms and random walk on the d-cube. Annals of Probability, 11:403–413, 1983.

[AW09] Gautham Anil and R. Paul Wiegand. Black-box search by elimination of fitness functions. In Proceedings of the 10th ACM Workshop on Foundations of Genetic Algorithms (FOGA’09), pages 67–78. ACM, 2009.

[BBD+09] Surender Baswana, Somenath Biswas, Benjamin Doerr, Tobias Friedrich, Piyush P. Kurur, and Frank Neumann. Computing single source shortest paths using single-objective fitness functions. In Proceedings of the 10th ACM Work- shop on Foundations of Genetic Algorithms (FOGA’09), pages 59–66, 2009. [CCH96] Zhixiang Chen, Carlos Cunha, and Steven Homer. Finding a hidden code by

asking questions. In Proceedings of the 2nd Annual International Conference on Computing and Combinatorics (COCOON’96), pages 50–55. Springer, 1996. [Chv83] Vasek Chvátal. Mastermind. Combinatorica, 3:325–329, 1983.

[CLTY09] Tianshi Chen, Per Kristian Lehre, Ke Tang, and Xin Yao. When is an estimation of distribution algorithm better than an evolutionary algorithm? In Proceedings of the 2009 IEEE Congress on Evolutionary Computation (CEC’09), pages 1470– 1477. IEEE, 2009.

[CTCY10] Tianshi Chen, Ke Tang, Guoliang Chen, and Xin Yao. Analysis of computa- tional time of simple estimation of distribution algorithms. IEEE Transactions on Evolutionary Computation, 14(1):1–22, Feb. 2010.

[DF11] Benjamin Doerr and Mahmoud Fouz. Asymptotically optimal randomized rumor spreading. In Proceedings of the 38th International Colloquium on Automata, Languages and Programming (ICALP’11), pages 502–513, 2011.

[DHK07] Benjamin Doerr, Edda Happ, and Christian Klein. A tight analysis of the (1+1)- EA for the single source shortest path problem. In Proceedings of the 2007 IEEE Congress on Evolutionary Computation (CEC’07), pages 1890–1895. IEEE, 2007. [DHK08] Benjamin Doerr, Edda Happ, and Christian Klein. Crossover can provably be useful in evolutionary computation. In Proceedings of the 10th Annual Genetic and Evolutionary Computation Conference (GECCO’08), pages 539–546. ACM, 2008.

160 Bibliography

[DJ10] Benjamin Doerr and Daniel Johannsen. Edge-based representation beats vertex- based representation in shortest path problems. In Proceedings of the 12th Annual Genetic and Evolutionary Computation Conference (GECCO’10), pages 758–766. ACM, 2010.

[DJK+11] Benjamin Doerr, Daniel Johannsen, Timo Kötzing, Per Kristian Lehre, Markus Wagner, and Carola Winzen. Faster black-box algorithms through higher arity operators. In Proceedings of the 11th ACM Workshop on Foundations of Genetic Algorithms (FOGA’11), pages 163–172. ACM, 2011.

[DJS+10] Benjamin Doerr, Thomas Jansen, Dirk Sudholt, Carola Winzen, and Christine Zarges. Optimizing monotone functions can be difficult. In Proceedings of the 11th International Conference on Parallel Problem Solving from Nature (PPSN’10), Part I, LNCS 6238, pages 42–51. Springer, 2010.

[DJTW03] Stefan Droste, Thomas Jansen, Karsten Tinnefeld, and Ingo Wegener. A new framework for the valuation of algorithms for black-box optimization. In Pro- ceedings of the 7th Workshop on Foundations of Genetic Algorithms (FOGA’03), pages 253–270. Morgan Kaufmann, 2003.

[DJW02] Stefan Droste, Thomas Jansen, and Ingo Wegener. On the analysis of the (1+1) evolutionary algorithm. Theoretical Computer Science, 276:51–81, 2002.

[DJW06] Stefan Droste, Thomas Jansen, and Ingo Wegener. Upper and lower bounds for randomized search heuristics in black-box optimization. Theory of Computing Systems, 39:525–544, 2006.

[DJW10a] Benjamin Doerr, Daniel Johannsen, and Carola Winzen. Drift analysis and lin- ear functions revisited. In Proceedings of 2010 IEEE Congress on Evolutionary Computation (CEC’10), pages 1967–1974. IEEE, 2010.

[DJW10b] Benjamin Doerr, Daniel Johannsen, and Carola Winzen. Multiplicative drift anal- ysis. In Proceedings of the 12th Annual Genetic and Evolutionary Computation Conference (GECCO’10), pages 1449–1456, 2010.

[DKLW11] Benjamin Doerr, Timo Kötzing, Johannes Lengler, and Carola Winzen. Black- box complexities of combinatorial problems. In Proceedings of the 13th Annual Genetic and Evolutionary Computation Conference (GECCO’11), pages 981–988. ACM, 2011.

[DKW11] Benjamin Doerr, Timo Kötzing, and Carola Winzen. Too fast unbiased black- box algorithms. In Proceedings of the 13th Annual Genetic and Evolutionary Computation Conference (GECCO’11), pages 2043–2050. ACM, 2011.

[DNHW03] Martin Dietzfelbinger, Bart Naudts, Clarissa Van Hoyweghen, and Ingo Wegener. The analysis of a recombinative hill-climber on H-IFF. IEEE Transactions on Evolutionary Computation, 7(5):417–423, Oct. 2003.

[DP09] Devdatt P. Dubhashi and Alessandro Panconesi. Concentration of Measure for the Analysis of Randomised Algorithms. Cambridge University Press, 2009. [DS04] Marco Dorigo and Thomas Stützle. Ant Colony Optimization. MIT Press, 2004. [DW11a] Benjamin Doerr and Carola Winzen. Breaking the O(n log n) barrier of Leading-

Ones. In Artificial Evolution (EA’11), 2011. Accepted for presentation.

[DW11b] Benjamin Doerr and Carola Winzen. Memory-restricted black-box complexity. ECCC TR11-092, 2011.

[DW11c] Benjamin Doerr and Carola Winzen. Towards a complexity theory of randomized search heuristics: Ranking-based black-box complexity. In Proceedings the 6th International Computer Science Symposium in Russia (CSR’11), pages 15–28. Springer, 2011.

[ER63] Paul Erdős and Alfréd Rényi. On two problems of information theory. Magyar Tud. Akad. Mat. Kutató Int. Közl., 8:229–243, 1963.

[FK86] Johannes B. G. Frenk and Alexander H. G. Rinnooy Kan. The rate of convergence to optimality of the LPT rule. Discrete Applied Mathematics, 14:187–197, 1986. [FW05] Simon Fischer and Ingo Wegener. The one-dimensional Ising model: Mutation

versus recombination. Theoretical Computer Science, 344(2-3):208–225, 2005. [GJ90] Michael R. Garey and David S. Johnson. Computers and Intractability; A Guide

to the Theory of NP-Completeness. W. H. Freeman & Co., 1990.

[Goo09] Michael T. Goodrich. On the algorithmic complexity of the mastermind game with black-peg results. Information Processing Letters, 109:675–678, 2009. [GSW09] Michael Gnewuch, Anand Srivastav, and Carola Winzen. Finding optimal vol-

ume subintervals with k points and calculating the star discrepancy are NP-hard problems. Journal of Complexity, 25:115–127, 2009.

[Gut07] Walter J. Gutjahr. Mathematical runtime analysis of ACO algorithms: Survey on an emerging issue. Swarm Intelligence, 1:59–79, 2007.

[GWW11] Michael Gnewuch, Magnus Wahlström, and Carola Winzen. A randomized algo- rithm based on threshold accepting to approximate the star discrepancy. CoRR, abs/1103.2102, 2011.

[Hro01] Juraj Hromkovič. Algorithmics for Hard Problems: Introduction to Combina- torial Optimization, Randomization, Approximation, and Heuristics. Springer, 2001.

[HY04] Jun He and Xin Yao. A study of drift analysis for estimating computation time of evolutionary algorithms. Natural Computing, 3:21–35, 2004.

[JW02] Thomas Jansen and Ingo Wegener. The analysis of evolutionary algorithms - a proof that crossover really can help. Algorithmica, 34:47–66, 2002.

[Kar72] Richard M. Karp. Reducibility among combinatorial problems. In Proceedings of a Symposium on the Complexity of Computer Computations, pages 85–103. Plenum Press, 1972.

[KE01] James Kennedy and Russell C. Eberhart. Swarm Intelligence. Morgan Kaufmann, 2001.

[KK09] Petr Kovár and Michael Kubesa. Factorizations of complete graphs into spanning trees with all possible maximum degrees. In Proceedings of the 20th International Workshop on Combinatorial Algorithms (IWOCA’09), pages 334–344. Springer, 2009.

[Knu77] Donald E. Knuth. The computer as a master mind. Journal of Recreational Mathematics, 9:1–5, 1977.

[KST11] Timo Kötzing, Dirk Sudholt, and Madeleine Theile. How crossover helps in pseudo-Boolean optimization. In Proceedings of the 13th Annual Genetic and Evolutionary Computation Conference (GECCO’11), pages 989–996, 2011.

162 Bibliography

[LL02] Pedro Larrañaga and José A. Lozano. Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Kluwer Academic Publishers, 2002. [LTT89] Donna Crystal Llewellyn, Craig Tovey, and Michael Trick. Local optimization on

graphs. Discrete Applied Mathematics, 23:157–178, 1989. Erratum: 46:93–94, 1993.

[LW10a] Per Kristian Lehre and Carsten Witt. Black-box search by unbiased variation. In Proceedings of the 12th Annual Genetic and Evolutionary Computation Confer- ence (GECCO’10), pages 1441–1448. ACM, 2010.

[LW10b] Per Kristian Lehre and Carsten Witt. Black-box search by unbiased variation. Electronic Colloquium on Computational Complexity (ECCC), page 16, 2010. [MR95] Rajeev Motwani and Prabhakar Raghavan. Randomized Algorithms. Cambridge

University Press, 1995.

[Müh92] Heinz Mühlenbein. How genetic algorithms really work: Mutation and hillclimb- ing. In Proceedings of the 2nd International Conference on Parallel Problem Solving from Nature (PPSN’92), pages 15–26. Elsevier, 1992.

[NW04] Frank Neumann and Ingo Wegener. Randomized local search, evolutionary al- gorithms, and the minimum spanning tree problem. In Proceedings of the 6th Annual Genetic and Evolutionary Computation Conference (GECCO’04), pages 713–724. Springer, 2004.

[NW07] Frank Neumann and Ingo Wegener. Randomized local search, evolutionary algo- rithms, and the minimum spanning tree problem. Theoretical Computer Science, 378:32–40, 2007.

[NW09] Frank Neumann and Carsten Witt. Runtime analysis of a simple ant colony optimization algorithm. Algorithmica, 54:243–255, 2009.

[NW10] Frank Neumann and Carsten Witt. Bioinspired Computation in Combinatorial Optimization – Algorithms and Their Computational Complexity. Springer, 2010. [OHY08] Pietro S. Oliveto, Jun He, and Xin Yao. Analysis of population-based evolutionary algorithms for the vertex cover problem. In Proceedings of IEEE World Congress on Computational Intelligence (WCCI 2008), Hong Kong, June 1-6, 2008, pages 1563–1570, 2008.

[Rob55] Herbert Robbins. A remark on Stirling’s formula. The American Mathematical Monthly, 62:26–29, 1955.

[RS09] Joachim Reichel and Martin Skutella. On the size of weights in randomized search heuristics. In Proceedings of the 10th ACM Workshop on Foundations of Genetic Algorithms (FOGA’09), pages 21–28. ACM, 2009.

[Rud97] Günter Rudolph. Convergence Properties of Evolutionary Algorithms. Kovac, 1997.

[RV11] Jonathan Rowe and Michael Vose. Unbiased black box search algorithms. In Pro- ceedings of the 13th Annual Genetic and Evolutionary Computation Conference (GECCO’11), pages 2035–2042. ACM, 2011.

[STW02] Jens Scharnow, Karsten Tinnefeld, and Ingo Wegener. Fitness landscapes based on sorting and shortest path problems. In Proceedings of the 7th International Conference on Parallel Problem Solving from Nature (PPSN’02), pages 54–63. Springer, 2002.

[STW04] Jens Scharnow, Karsten Tinnefeld, and Ingo Wegener. The analysis of evolution- ary algorithms on sorting and shortest paths problems. Journal of Mathematical Modelling and Algorithms, pages 349–366, 2004.

[Sud05] Dirk Sudholt. Crossover is provably essential for the Ising model on trees. In Pro- ceedings of the 7th Annual Genetic and Evolutionary Computation Conference (GECCO’05), pages 1161–1167, 2005.

[SW04] Tobias Storch and Ingo Wegener. Real royal road functions for constant popula- tion size. Theoretical Computer Science, 320(1):123–134, 2004.

[SZ06] Jeff Stuckman and Guo-Qiang Zhang. Mastermind is NP-complete. INFOCOMP Journal of Computer Science, 5:25–28, 2006.

[Win11] Carola Winzen. Direction-reversing quasi-random rumor spreading with restarts. CoRR, abs/1103.2429, 2011.

[Wit05] Carsten Witt. Worst-case and average-case approximations by simple randomized search heuristics. In Proceedings of the 22nd Annual Symposium on Theoretical Aspects of Computer Science (STACS’05), pages 44–56. Springer, 2005.

[Yao77] Andrew Chi-Chin Yao. Probabilistic computations: Toward a unified measure of complexity. In Proceedings of the 18th Annual Symposium on Foundations of Computer Science (FOCS’77), pages 222–227, 1977.

[Zha06] Shengyu Zhang. New upper and lower bounds for randomized and quantum local search. In Proceedings of the 38th Annual ACM Symposium on Theory of Computing (STOC’06), pages 634–643. ACM, 2006.

A

Further Contributions

A.1. Theory of Randomized Search Heuristics

Multiplicative Drift Analysis

In this work we introduce multiplicative drift analysis as a suitable way to analyze the runtime of randomized search heuristics such as evolutionary algorithms.

We give a multiplicative version of the classical drift theorem. This allows easier analyses in those settings where the optimization progress is roughly proportional to the current distance to the optimum.

To display the strength of this tool, we regard the classical problem of how the (1+1) evolutionary algorithm optimizes an arbitrary linear pseudo-Boolean function. Here, we first give a relatively simple proof for the fact that any linear function is optimized in expected number of O(n log n) function evaluations, where n is the length of the bit string. Afterwards, we show that in fact any such function is optimized in expected time at most (1 + o(1))1.39en log n, again using multiplicative drift analysis. We also prove a corresponding lower bound of (1 − o(1))en log n which actually holds for all functions with a unique global optimum.

We further demonstrate how our drift theorem immediately gives natural proofs (with better constants) for the best known runtime bounds for the (1+1) EA on combinatorial problems like finding minimum spanning trees, shortest paths, or Euler tours in graphs.

In: Proceedings of Gecco ’10 (cf. [DJW10b]). Invited and submitted to a special issue of Algorithmica.

166 Further Contributions

Non-Existence of Linear Universal Drift Functions

Drift analysis is a powerful tool to prove upper and lower bounds on the runtime of randomized search heuristics. Its most famous application is a simple proof for the classical problem how the (1+1) evolutionary algorithm optimizes linear pseudo- Boolean functions. A relatively simple potential function allows to track the progress of the EA optimizing any linear function.

In this work, we show that such beautiful proofs cease to exist if the mutation probability is slightly larger than the standard value of 1/n. In fact, we show that no universal liner drift function exists once we increase the mutation probability to c/n for some small constant c > 1.

In: Proceedings of Gecco ’10 and CEC ’10 (cf. [DJW10b, DJW10a]). A journal version is currently under submission.

Mutation Rate Matters Even When Optimizing Monotone Functions

Extending previous analyses on function classes like linear functions, we analyze how the (1+1) evolutionary algorithm optimizes pseudo-Boolean functions that are strictly monotone. These functions have the property that whenever only 0-bits are changed to 1, then the objective value strictly increases. Contrary to what one would expect, not all of these functions are easy to optimize. The choice of the constant c in the mutation probability p(n) = c/n can make a decisive difference.

We show that if c < 1, then the (1+1) EA finds the optimum of every such function in Θ(n log n) iterations. For c = 1, we can still prove an upper bound of O(n3/2). However, for c ≥ 16, we present a strictly monotone function such that the (1+1) EA with overwhelming probability needs 2Ω(n) iterations to find the optimum. This is the first time that we observe that a constant factor change of the mutation probability changes the run-time by more than constant factors.

In: Proceedings of PPSN ’10 (cf. [DJS+10]). A journal version is currently under submission.