Ant Colony Optimization

Ant colony optimization (ACO) is another important bio-inspired approach to solve opti-mization problems. ACO algorithms are inspired by the search of an ant colony for a common source of food. It has been noticed that ants find very quickly a shortest path to a source of food. The information about which way to take to get to the food is distributed between the ants by leaving an information, called pheromone, on the way an ant has taken. As longer paths to the source take much more time than shorter paths, shorter paths are more often visited. This implies larger pheromone values on shorter paths after a small amount of time.

In contrast to evolutionary algorithms where solutions are constructed from the current set of solutions, ACO algorithms obtain new solutions by random walks on a so-called construc-tion graph. These random walks are influenced by the pheromone values of the edges of the underlying construction graph. Edges that correspond to ”good” partial solutions should receive large pheromone values during the optimization process such that good solutions are obtained. Our goal is to understand the random process behind the optimization of ACO algorithms.

Update Schemes

Investigators: Benjamin Doerr, Daniel Johannsen, and Frank Neumann in cooperation with Dirk Sudholt (TU Dortmund) and Carsten Witt (DTU Copenhagen)

Regarding ACO, only convergence results [4] were known until 2006 and analyzing the run-time of ACO algorithms has been pointed out as a challenging task in [3]. Basic ant colony

optimization (ACO) algorithms are governed by a single main parameter, the evaporation factor. Such algorithms successively generate candidate solutions to an optimization problem according to a distribution based on the solutions generated so far. The evaporation factor determines how fast the influence of past solutions decreases as new and better solutions are generated. First steps into analyzing the runtime of ACO algorithms have been made in [5], and, independently, the first runtime analyses of a simple ACO algorithm called 1-ANT were done at the same time in [8]. Subsequently this algorithm was further investigated for the optimization of some well-known pseudo-Boolean functions [2]. In [1], we simplified the view on these problems by an appropriate translation of the underlying model. This results in a profound simplification of the involved equations and a nonlinear rescaling of the evaporation factor. Investigating the rescaled evaporation factor allowed us to refine the results given in [8]. In particular, we showed how the exponential runtime bound gradually changes to a polynomial bound inside the window of phase transition. A conclusion of the mentioned investigations is that 1-ANT is very sensitive to the choice of the evaporation factor ρ. Decreasing the value of ρ only by a small amount may lead to a phase transition and turn a polynomial runtime into an exponential one.

Many ACO algorithms used in applications use best-so-far reinforcement where in every iteration the current best-so-far solution is reinforced. In other words, in every iteration a pheromone update happens, using either the old or a newly generated best-so-far solution.

We showed in [7] how these algorithms, variants of the MAX-MIN ant system (MMAS) (see [9]), can be analyzed for various example functions, including the class of unimodal functions and plateau functions. Thereby, we extended previous work by Gutjahr and Sebastiani [6].

The latter authors first analyzed MMAS variants on OneMax, LeadingOnes, and functions with plateaus. Their results and our contributions show that the impact of ρ is by far not as drastic as for the 1-ANT. When decreasing ρ, the algorithms become more and more similar to random search and the runtime on simple functions grows with 1/ρ, but there is no phase transition for polynomially small ρ as for the 1-ANT. We also demonstrated how (a restricted formulation of) the fitness-level method can be adapted to the analysis of ACO algorithms. Finally, we presented lower bounds for ACO algorithms: a general lower bound for functions with unique optimum that grows with 1/ρ and an almost tight lower bound for LeadingOnes.

References

• [1] B. Doerr and D. Johannsen. Refined runtime analysis of a basic ant colony optimization algorithm.

In IEEE Congress on Evolutionary Computation 2007, Singapore, 2007, pp. 501–507. IEEE.

• [2] B. Doerr, F. Neumann, D. Sudholt, and C. Witt. On the runtime analysis of the 1-ANT ACO algorithm. In D. Thierens, ed., Genetic and Evolutionary Computation Conference 2007, London, UK, 2007, pp. 33–40. ACM. Best paper award.

[3] M. Dorigo and C. Blum. Ant colony optimization theory: A survey. Theoretical Computer Science, 344:243–278, 2005.

[4] W. J. Gutjahr. ACO algorithms with guaranteed convergence to the optimal solution. Information Processing Letters, 82(3):145–153, 2002.

[5] W. J. Gutjahr. First steps to the runtime complexity analysis of Ant Colony Optimization.

Computers and Operations Research, 35(9):2711–2727, 2008.

0 200 400 600 800 1000

0 2000 4000 6000 8000 10000 12000 14000 16000 t

ρ = 0.2 (mm

---: 988.227, P

--: 0.20597, ν

--: 1.90696)

f_max µν P mm

Figure 2.1: A typical run of an ACO algorithm. We track indicators of the optimization behavior like the probability to find a new solution, variance and expectation of the next solutions value, and the value of the best solution so far.

[6] W. J. Gutjahr and G. Sebastiani. Runtime analysis of ant colony optimization with best-so-far reinforcement. Methodology and Computing in Applied Probability, 10:409–433, 2008.

• [7] F. Neumann, D. Sudholt, and C. Witt. Analysis of different MMAS ACO algorithms on unimodal functions and plateaus. Swarm Intelligence, 3(1):35–68, 2009.

• [8] F. Neumann and C. Witt. Runtime analysis of a simple ant colony optimization algorithm.

Algorithmica, 2009. To appear.

[9] T. St¨utzle and H. H. Hoos. MAX-MIN ant system. Journal of Future Generation Computer Systems, 16:889–914, 2000.

Experimental Analysis of the Optimization Behavior of Single Ant ACO Systems Investigators: Benjamin Doerr, Daniel Johannsen, and Ching Hoo Tang

In [1], we undertook a rigorous experimental analysis of the optimization behavior of the two most studied single ant ACO systems: 1–Ant and the Max–Min Ant System (MMAS).

By tracking the behavior of the underlying random processes rather than just regarding the resulting optimization time, we gained additional in sight into these systems.

These experiments are motivated and guided by preceding theoretical runtime analyses of the considered ACO systems on pseudo-boolean optimization problems. Although both, algorithms and problems, are rather basic, the rigorous theoretical analyses and especially the mathematical methods applied therein are far from simple.

To complement these theoretical results, we conducted an experimental analysis of these two single ant ACO systems on several pseudo-boolean fitness functions (onemax,

leadin-gones, and linear functions with random weights). To gain an understanding how these algorithms work, we tracked a number of theory–guided indicators (other than the resulting optimization time) during the runs of 1–Ant and the MMAS.

It turned out that in those cases where one of the two ACO system performs well, it basically simulates the much simpler (1+1) evolutionary algorithm. This shows that the pessimistic assumptions repeatedly used in the proofs of the results mentioned above are real, and in consequence, indicates that the upper bounds on the optimization time proven there probably cannot be improved. Our analysis of the optimization behavior fits well to the fact that we rarely observed that one of the two single ant ACO systems finds the optimum significantly faster than the (1+1) EA.

References

• [1] B. Doerr, D. Johannsen, and C. H. Tang. How single ant aco systems optimize pseudo-boolean functions. In Parallel Problem Solving from Nature ? PPSN X, USA, Atlanta, 2008, LNCS 5199, pp. 378–388. Springer.

Hybridization with Local Search

Investigators: Frank Neumann in cooperation with Dirk Sudholt (TU Dortmund) and Carsten Witt (DTU Copenhagen)

Often successful applications of ACO use a combination with local search procedures that improve the solutions constructed by the ants. The effect of using local search with ACO algorithms is manifold. Firstly, local search can help to find good solutions more quickly as it increases the “greediness” within the algorithm. Similar to memetic evolutionary algorithms, local search can also be used to discover the real “potential” of a solution as it can turn a bad looking solution into a good local optimum. Moreover, the pivot rule used in local search may guide the algorithm towards certain regions of the search space.

There is another effect that we investigated more closely in [1]. The pheromone values induce a sampling distribution over the search space. On a typical fitness landscape, once the best-so-far solution has reached a certain quality, sampling new solutions with a high variance becomes inefficient and the current best-so-far solution x^∗ is maintained for some time. Our analyses presented in [2] have shown that then the pheromones quickly reach the upper and lower bounds corresponding to x^∗. This means that the algorithm turns to sampling close to x^∗. In other words, simple ACO algorithms typically reach a situation where the “center of gravity” of the sampling distribution follows the current best-so-far solution and the variance of the sampling distribution is low.

When introducing local search into ACO algorithm, this may not be true. Local search is able to find local optima that are far away from the current best-so-far solution. In this case the “center of gravity” of the sampling distribution is far away from the best-so-far solution.

We have presented simple functions where the behavior of simple ACO algorithms with and without local search have a different runtime behavior. We proved exponential runtime bounds that holds with probability exponentially close to 1 for ACO algorithms not using local search and polynomial bounds for the algorithms hybridizing ACO and local search.

Our analyses exploit that the sampling distributions can follow different routes through the search space which leads to the different behavior of the two algorithmic approaches.

References

• [1] F. Neumann, D. Sudholt, and C. Witt. Rigorous analyses for the combination of ant colony optimization and local search. In M. Dorigo, M. Birattari, C. Blum, M. Clerc, T. St¨utzle, and A. F. T. Winfield, eds., International Conference on Ant Colony Optimization and Swarm Intel-ligence 2008, Brussels, Belgium, 2008, LNCS 5217, pp. 132–143. Springer.

• [2] F. Neumann, D. Sudholt, and C. Witt. Analysis of different MMAS ACO algorithms on unimodal functions and plateaus. Swarm Intelligence, 3(1):35–68, 2009.