Ant Colony Optimization (ACO)

Ant Colony Optimization (ACO)

• Exploits foraging behavior of ants

– Path optimization

• Problems mapping onto “foraging” are ACO-like

– TSP, ATSP – QAP

Travelling Salesman Problem (TSP)

• Why?

– Hard, shortest path problem

– NP hard, in fact (no polynomial time algorithm) – Well studied (can compare results to those

known)

• GA [Freisleben and Merz]

• Iterated Lin-Kernighan etc.

– Easy to explain

TSP

Connect n points,

visiting points only once such that total path

length is minimized. Path is closed; i.e. start point and end point are same.

It isn’t easy … Symmetric: d_ij = d_ji

Asymmetric: d_ij ≠ d_ji

TSP

It’s easy to create non- optimal tours …

Ant System (AS) for TSP

• Ants build tours by moving on graph

– Add cities one at a time until tour is complete

• Transition from node i to j depends on:

– Whether node visited (ants have a tabu list) – Inverse of distance,

– Pheromone deposits, – Transition rule:

ij

ij = 1/ d

η

τ

[ ] [ ]

[ ] [ ]

∑

=

ik

J l

il il

ij k ij

ij t

p _{α β}

β α

η τ

η ) τ

(

Transition Equation

is the tabu list

Constants: α, β

α = 0, no cooperation, greedy search β = 0 no preference for shortest edge Note: p_ij^k(t), even though pheromone concentrations same, tabu list may be different

[ ] [ ]

[ ] [ ]

∑

=

ik

J l

il il

ij k ij

ij t

p _{α β}

β α

η τ

η ) τ

k (

J

Pheromone Laying

• When tour complete, pheromone updated:

• T^k(t) tour by k^th ant for iteration t

• Q a system parameter, value is relatively unimportant





∉

= ∈

∆ 0 ( , ) ( )

) ( )

, ( )

( ) /

( if i j T t

t T

j i if t

L

t Q _k

k k

k

τij

Pheromone Decay

• System performs poorly without pheromone decay:

– Saturates, local optimum found

• Pheromone decay:

) ( )

( ).

1 ( )

( t

t

t

ij

ρ τ τ

τ ←  − + ∆

∑

∆

=

∆

k

k ij

ij

t t

1

) ( )

( τ

τ

τ_ij(0) small

Ant Density and Initial Positioning

• Number of ants, m, is constant for run:

– Too many, reinforcement too rapid

– Too few, pheromone decay prevents trail formation

• m = n found to be reasonable

• Initial positioning:

– One per node – Random

Elitest Ants

• Stolen from Genetic Algorithms (GAs)

– Elitist ant is one which reinforces edges belonging to best tour, T⁺ with length L⁺

– Each iteration, e elitist ants are added to other ants so that edges in T⁺ get extra e.Q/L⁺

• Improves algorithm … slightly

Complexity and Results

• Time complexity: O(t.n².m) = O(t.n³), n=m

– T = # iterations, n = # cities, m = # ants

• Initial results:

– Disappointing, did not match Lin-Kernighan – Better for small TSP (Oliver 30 city)

– Rapid convergence, but only fair for 70 cities

Alg 2-1 (top)

Ant initialization

Next Edge Choice

Tour Length Calculation Best tour update

Swarm Intelligence: Bonabeau et al

Alg 2-1 (bottom)

Ant pheromone updates

k^th Ant updates Elitest Ant

updates

Save new edge Pheromone values

Swarm Intelligence: Bonabeau et al

Performance Results

25.1 459.8

422 SA

1.5 420.6

420 TS

1.3 420.4

420 AS-TSP

Std Dev.

Average Best Tour

Oliver30

best

Figure 2.12

Rapid Convergence for best tour:

1. Tends to avoid getting trapped in local optima 2. Maintains diversity in solutions as branching factor > 2

Swarm Intelligence: Bonabeau et al

Improvements

• Another transition rule:

[ ] [ ]

{ }





>

= ^∈ ≤

0

. 0

) ( max

arg

q q

if J

q q

if

j ^{u Jik iu} t ^iu

η β

τ

[ ] [ ]

[ ] [ ]

∑

=

ik

J

l il il

iJ k iJ

iJ t

t t

p _β

β

η τ

η τ

) ( ) ) (

(

q uniformly distributed [0,1], q₀ is tunable

where J

J ∈ _i^k ,

Observations

• If q > q₀, same transition as before

• If q ≤ q₀, make “greedy” choice

– Uses heuristic knowledge about problem:

distances and pheromone trail

• q₀, ~1 choose locally optimal solutions

• q₀, ~0 all locally optimal solutions

evaluated Apply Simulated Annealing

to q₀

Figure 2-13

Swarm Intelligence: Bonabeau et al

Maintains diversity –

Standard deviation remains “high”

Figure 2-14

Swarm Intelligence: Bonabeau et al

Considerable Diversity Remains during run Little initial pheromone,

choose all edges with same probability

Improved Pheromone Updating

• All ants deposit pheromone on tour completion

• ACS, best ant since beginning of trial updates network

– Encourages ants to search within vicinity of best tour found

– Exploration is more directed, more focussed

• Pheromone updates applied ONLY to edges on best path, (i,j) edges belonging to T⁺

Pheromone Updates

• Two forms:

– During creation of tour

– Upon tour completion by all ants

Pheromone Trail Updates

( ¹ ) ^{. ( ) . ( )}

)

( t

t

t

ij

ρ τ ρ τ

τ ←  − + ∆

= +

∆ τ _ij ( t ) 1 / L

For Each Ant Transition:

Local Pheromone Updates

( ¹ ) ^{. ( ) .}

)

( ρ τ ρ τ

τ

t ←  −

t +

τ₀ = initial value on pheromone trail

Set up to be 1/(n.L_nn), L_nn = length of tour found by nearest neighbor (NN) heuristic

When ant visits an edge, the pheromone concentration

decreases, making them less attractive forcing exploration of not yet visited edges.

Effect of local updating is to “shuffle tours.” Edge desirability changes dynamically as ants visit them.

Run NN demo

Candidate Lists

• Maintain list of preferred cities to be visited from a given city (size cl), ordered by distance

• Unvisited cities examined first, next city chosen according to:

[ ] [ ]

{ }





>

= ^∈ ≤

0

. 0

) ( max

arg

q q

if J

q q

if

j ^{u Jik iu} t ^iu

η β

τ

[ ] [ ]

[ ] [ ]

∑

=

ik

J

l il il

iJ k iJ

iJ t

t t

p _β

β

η τ

η τ

) ( ) ) (

(

where J

J ∈ _i^k ,

ACS-TSP

Swarm Intelligence: Bonabeau et al

Edge initialization

Ant initialization

Compute tour length

Tour Length Calculation Next Edge

Choice

Local Update

Alg 2-2a

Swarm Intelligence: Bonabeau et al

Best tour update

Ant pheromone Updates for

best tour Save new edge Pheromone values

Results (25 runs)

6.70 6.49

6.33 6.18

50 City (V)

5.87 5.86

5.81 5.74

50 City (IV)

5.83 5.70

5.65 5.58

50 City (III)

6.25 6.03

6.01 6.05

50 City (II)

6.06 5.89

5.88 5.88

50 City (I)

SOM EN

SA ACS-

TSP

best

Results

N/A N/A

(N/A) N/A

N/A (N/A) 103000

21761 (N/A) 4820

21282 (21285.44) KroA100

(100 cities)

173250 580

(N/A) 325000

542 (549.18) 80000

545 (N/A) 3480

535 (542.37) Eil75

(75 cities)

68512 443

(N/A) 100000

426 (427.86) 25000

428 (N/A) 1830

425 (427.96) Eil50

(50 cities)

SA

#iter SA

best EP

#iter EP

best GA

#iter GA

best ACS-

TSP

#iter ACS-TSP

best

best

Adding Local Heuristics

• To scale, add local optimization:

– Works in combination with ACS

– For TSP, use 2-opt, 3-opt and Lin-Kernighan

Run 2-opt demo Eliminate 2 edges and

reconnect two resulting paths to generate a different tour

ACS-3-opt

• Same as ACS, except:

– Each ant tour is minimized upon tour creation

For k = 1 to m do

T^k(t) ← 3-opt(T^k(t)) {apply local opt to each tour}

End For

ACS-3-opt Results (sTSP)

8807.3 8806

8837.9 8818

rat783 (783 cities)

27693.7 27686

27718.2 27693

att532 (532 cities)

42029 42029

42029 42029

lin318 (318 cities)

15780 15780

15781.7 15780

d198

(198 cities)

STSP (GA) Average STSP (GA)

Best ACS-3-opt

Average ACS-3-opt

Best

best

ACS-3-opt Results (aTSP)

2766.1 2755

2755 2755

ftv170 (170 cities)

36235.3 36230

36230 36230

kro124p (124 cities)

38683.8 38673

38679.8 38673

ft70

(70 cities)

14440 14422

14422 14422

ry48p (48 cities)

2810 2810

2810 2810

p43

(43 cities)

ATSP Average ATSP

Best ACS-3-opt

Average ACS-3-opt

Best

best

Other Potential Improvements

• All r best ants to update trail, instead of single ant

– Reduce probability of being trapped in local opt.

• Allow β – pheromone sensitivity -- to vary

• Remove pheromone from edges belonging to worst tours; i.e. those below average

– Increase convergence speed

• Use improved local search

Several potential projects

Observations

• Pheromone concentrations vary on edges

– Some strongly marked, some weakly – Strong on tour (likely)

– Weak point to alternative solutions

• Diversity of solutions useful in a dynamic environment

– We don’t start from scratch Swarm may be Superior in a

Dynamic environment

Other methods

• Min-Max AS [Stützle and Hoos]

– Iteration’s best ant updates pheromone trail

– Pheromone concentrations limited to [τ_min,τ_max] – Trails initialized to τ_max

• Bounding concentrations prevents stagnation

• Also,

– Strong trails reinforced less than weak

• Results better than AS, comparable with ACS

(

)

)

(t _{max ij} t

ij

τ τ

τ

∆

Elitest Mechanisms Bullnheimer et al

• Sort m ants according to tour length, Lⁱ(t)

• Ants update edges based upon their rank

• Used σ elitest ants:

– best σ – 1 deposit trail

– Weighted according to max {0, σ – µ}

( )

( )^{. / ( ), if ant uses (i, j), 0 otherwise}

) (

, ) ( )

(

) ( /

) (

) ( )

( .

) ( . 1

) (

1

1

µ µ

σ τ

τ τ

τ

τ τ

σ τ

ρ τ

µ µ

σ µ

µ

t L Q t

t t

t L Q t

where

t t

t t

ij

ij r

ij ij

ij ij

ij ij

−

=

∆

∆

=

∆

=

∆

∆ +

∆ +

−

 

←

∑⁻

= + +

+