Scatter Search, SS.
2.9.1
Taxonomy
Scatter search is a Metaheuristic and a Global Optimization algorithm. It is also sometimes associated with the field of Evolutionary Computation given the use of a population and recombination in the structure of the technique. Scatter Search is a sibling of Tabu Search (Section2.10), developed by the same author and based on similar origins.
2.9.2
Strategy
The objective of Scatter Search is to maintain a set of diverse and high- quality candidate solutions. The principle of the approach is that useful information about the global optima is stored in a diverse and elite set of solutions (the reference set) and that recombining samples from the set can exploit this information. The strategy involves an iterative process, where a population of diverse and high-quality candidate solutions that are partitioned into subsets and linearly recombined to create weighted centroids of sample-based neighborhoods. The results of recombination are refined using an embedded heuristic and assessed in the context of the reference set as to whether or not they are retained.
2.9.3
Procedure
Algorithm2.9.1provides a pseudocode listing of the Scatter Search algorithm for minimizing a cost function. The procedure is based on the abstract form presented by Glover as a template for the general class of technique [3], with influences from an application of the technique to function optimization by Glover [3].
2.9.4
Heuristics
Scatter search is suitable for both discrete domains such as combina- torial optimization as well as continuous domains such as non-linear programming (continuous function optimization).
Small set sizes are preferred for the ReferenceSet, such as 10 or 20 members.
Subset sizes can be 2, 3, 4 or more members that are all recombined to produce viable candidate solutions within the neighborhood of the members of the subset.
2.9. Scatter Search 67 Algorithm 2.9.1: Pseudocode for Scatter Search.
Input: DiverseSetsize, Ref erenceSetsize Output: ReferenceSet InitialSet← ConstructInitialSolution(DiverseSetsize); 1 RefinedSet← ∅; 2 forSi ∈ InitialSet do 3 RefinedSet← LocalSearch(Si); 4 end 5
ReferenceSet← SelectInitialReferenceSet(Ref erenceSetsize); 6 while¬ StopCondition() do 7 Subsets← SelectSubset(ReferenceSet); 8 CandidateSet← ∅; 9
forSubseti ∈ Subsets do 10
RecombinedCandidates← RecombineMembers(Subseti); 11 forSi ∈ RecombinedCandidates do 12 CandidateSet← LocalSearch(Si); 13 end 14 end 15
ReferenceSet← Select(ReferenceSet, CandidateSet, 16 Ref erenceSetsize); end 17 returnReferenceSet; 18
Each subset should comprise at least one member added to the set in the previous algorithm iteration.
The Local Search procedure should be a problem-specific improvement heuristic.
The selection of members for the ReferenceSet at the end of each iteration favors solutions with higher quality and may also promote diversity.
The ReferenceSet may be updated at the end of an iteration, or dynamically as candidates are created (a so-called steady-state popu- lation in some evolutionary computation literature).
A lack of changes to the ReferenceSet may be used as a signal to stop the current search, and potentially restart the search with a newly initialized ReferenceSet.
2.9.5
Code Listing
Listing2.8provides an example of the Scatter Search algorithm implemented in the Ruby Programming Language. The example problem is an instance of
a continuous function optimization that seeks min f (x) where f =!ni=1x2 i, −5.0 ≤ xi≤ 5.0 and n = 3. The optimal solution for this basin function is (v1, . . . , vn) = 0.0.
The algorithm is an implementation of Scatter Search as described in an application of the technique to unconstrained non-linear optimization by Glover [6]. The seeds for initial solutions are generated as random vectors, as opposed to stratified samples. The example was further simplified by not including a restart strategy, and the exclusion of diversity maintenance in the ReferenceSet. A stochastic local search algorithm is used as the embedded heuristic that uses a stochastic step size in the range of half a percent of the search space.
1 objective_function(vector)
2 vector.inject(0) {|sum, x| sum + (x ** 2.0)} 3
4
5 rand_in_bounds(min, max) 6 min + ((max-min) * rand()) 7 8 9 random_vector(minmax) 10 Array.new(minmax.size) |i| 11 rand_in_bounds(minmax[i][0], minmax[i][1]) 12 13 14
15 take_step(minmax, current, step_size) 16 position = Array.new(current.size) 17 position.size.times |i|
18 min = [minmax[i][0], current[i]-step_size].max 19 max = [minmax[i][1], current[i]+step_size].min 20 position[i] = rand_in_bounds(min, max)
21
22 position
23 24
25 local_search(best, bounds, max_no_improv, step_size) 26 count = 0
27
28 candidate = {:vector=>take_step(bounds, best[:vector], step_size)} 29 candidate[:cost] = objective_function(candidate[:vector])
30 count = (candidate[:cost] < best[:cost]) ? 0 : count+1 31 best = candidate candidate[:cost] < best[:cost] 32 count >= max_no_improv
33 best
34 35
36 construct_initial_set(bounds, set_size, max_no_improv, step_size) 37 diverse_set = []
38
39 cand = {:vector=>random_vector(bounds)}
40 cand[:cost] = objective_function(cand[:vector])
41 cand = local_search(cand, bounds, max_no_improv, step_size)
2.9. Scatter Search 69 43 diverse_set.size == set_size 44 diverse_set 45 46 47 euclidean_distance(c1, c2) 48 sum = 0.0
49 c1.each_index {|i| sum += (c1[i]-c2[i])**2.0}
50 Math.sqrt(sum)
51 52
53 distance(v, set)
54 set.inject(0){|s,x| s + euclidean_distance(v, x[:vector])} 55
56
57 diversify(diverse_set, num_elite, ref_set_size) 58 diverse_set.sort!{|x,y| x[:cost] <=> y[:cost]} 59 ref_set = Array.new(num_elite){|i| diverse_set[i]} 60 remainder = diverse_set - ref_set
61 remainder.each{|c| c[:dist] = distance(c[:vector], ref_set)} 62 remainder.sort!{|x,y| y[:dist]<=>x[:dist]}
63 ref_set = ref_set + remainder.first(ref_set_size-ref_set.size) 64 [ref_set, ref_set[0]]
65 66
67 select_subsets(ref_set)
68 additions = ref_set.select{|c| c[:new]} 69 remainder = ref_set - additions
70 remainder = additions remainder. ? remainder.empty? 71 subsets = []
72 additions.each |a|
73 remainder.each{|r| subsets << [a,r] a!=r && !subsets.include?([r,a])} 74 75 subsets 76 77 78 recombine(subset, minmax) 79 a, b = subset 80 d = rand(euclidean_distance(a[:vector], b[:vector]))/2.0 81 children = [] 82 subset.each |p| 83 step = (rand<0.5) ? +d : -d 84 child = {:vector=>Array.new(minmax.size)} 85 child[:vector].each_index |i|
86 child[:vector][i] = p[:vector][i] + step
87 child[:vector][i]=minmax[i][0] child[:vector][i]<minmax[i][0] 88 child[:vector][i]=minmax[i][1] child[:vector][i]>minmax[i][1] 89 90 child[:cost] = objective_function(child[:vector]) 91 children << child 92 93 children 94 95
96 explore_subsets(bounds, ref_set, max_no_improv, step_size) 97 was_change =
99 ref_set.each{|c| c[:new] = } 100 subsets.each |subset|
101 candidates = recombine(subset, bounds) 102 improved = Array.new(candidates.size) |i|
103 local_search(candidates[i], bounds, max_no_improv, step_size) 104
105 improved.each |c|
106 !ref_set.any? {|x| x[:vector]==c[:vector]} 107 c[:new] =
108 ref_set.sort!{|x,y| x[:cost] <=> y[:cost]} 109 c[:cost] < ref_set.last[:cost]
110 ref_set.delete(ref_set.last) 111 ref_set << c
112 puts " >> added, cost=#{c[:cost]}"
113 was_change = 114 115 116 117 118 was_change 119 120
121 search(bounds, max_iter, ref_set_size, div_set_size, max_no_improv,
step_size, max_elite)
122 diverse_set = construct_initial_set(bounds, div_set_size, max_no_improv,
step_size)
123 ref_set, best = diversify(diverse_set, max_elite, ref_set_size) 124 ref_set.each{|c| c[:new] = }
125 max_iter.times |iter|
126 was_change = explore_subsets(bounds, ref_set, max_no_improv, step_size) 127 ref_set.sort!{|x,y| x[:cost] <=> y[:cost]}
128 best = ref_set.first ref_set.first[:cost] < best[:cost] 129 puts " > iter=#{(iter+1)}, best=#{best[:cost]}"
130 !was_change 131 132 best 133 134 135 == 0 136 # problem configuration 137 problem_size = 3
138 bounds = Array.new(problem_size) {|i| [-5, +5]} 139 # algorithm configuration 140 max_iter = 100 141 step_size = (bounds[0][1]-bounds[0][0])*0.005 142 max_no_improv = 30 143 ref_set_size = 10 144 diverse_set_size = 20 145 no_elite = 5
146 # execute the algorithm
147 best = search(bounds, max_iter, ref_set_size, diverse_set_size,
max_no_improv, step_size, no_elite)
148 puts "Done. Best Solution: c=#{best[:cost]}, v=#{best[:vector].inspect}" 149
2.9. Scatter Search 71
2.9.6
References
Primary Sources
A form of the Scatter Search algorithm was proposed by Glover for integer programming [1], based on Glover’s earlier work on surrogate constraints. The approach remained idle until it was revisited by Glover and combined with Tabu Search [2]. The modern canonical reference of the approach was proposed by Glover who provides an abstract template of the procedure that may be specialized for a given application domain [3].
Learn More
The primary reference for the approach is the book by Laguna and Mart´ı that reviews the principles of the approach in detail and presents tutorials on applications of the approach on standard problems using the C programming language [7]. There are many review articles and chapters on Scatter Search that may be used to supplement an understanding of the approach, such as a detailed review chapter by Glover [4], a review of the fundamentals of the approach and its relationship to an abstraction called ‘path linking’ by Glover, Laguna, and Mart´ı [5], and a modern overview of the technique by Mart´ı, Laguna, and Glover [8].
2.9.7
Bibliography
[1] F. Glover. Heuristics for integer programming using surrogate constraints. Decision Sciences, 8(1):156–166, 1977.
[2] F. Glover. Tabu search for nonlinear and parametric optimization (with links to genetic algorithms). Discrete Applied Mathematics, 49:231–255, 1994.
[3] F. Glover. Artificial Evolution, chapter A Template For Scatter Search And Path Relinking, page 13. Sprinter, 1998.
[4] F. Glover. New Ideas in Optimization, chapter Scatter search and path relinking, pages 297–316. McGraw-Hill Ltd., 1999.
[5] F. Glover, M. Laguna, and R. Mart´ı. Fundamentals of scatter search and path relinking. Control and Cybernetics, 39(3):653–684, 2000. [6] F. Glover, M. Laguna, and R. Mart´ı. Advances in Evolutionary Compu-
tation: Theory and Applications, chapter Scatter Search, pages 519–537. Springer-Verlag, 2003.
[7] M. Laguna and R. Mart´ı. Scatter search: methodology and implementa- tions in C. Kluwer Academic Publishers, 2003.
[8] R. Mart´ı, M. Laguna, and F. Glover. Principles of scatter search. European Journal of Operational Research, 169(1):359–372, 2006.