Part II MLS Foundations
5.5 Other trajectory methods
This chapter provides a detailed review of the main concepts on which most trajectory-based metaheuristics are based. The metaheuristics are grouped by their primary mode of operation: iteration, thresholding, memory, and other assorted concepts. Suggestions for how these concepts would be implemented under the MLS framework are presented and discussed.
5.1
Ascent Search
The most basic form of a local search heuristic is Ascent Search (Descent Search for minimization problems). This method, also known as the Hill Climbing heuristic, First Improving Local Search, Depth-First Search or simply Local Search, starts with a solution and moves to the first improving solution examined from the neighbourhood, continuing until it is stuck in a local optimum, i.e. there are no improving solutions in the neighbourhood, and then stops.
Steepest Ascent (also known as Best Improving Local Search or Breadth-First Search), is similar to
Ascent Search except that the whole neighbourhood is examined and the best improving neighbour is
selected. It can result in a shorter trajectory to a local optimum, but increases the processing time for each iteration.
Ascent Search has the tendency to get stuck in a local maximum of the underlying search topology. Depending on the problem, and the shape of the topology, it can still sometimes be reasonably effective. For example, if the neighbourhood scheme is such that every solution can be reached within one transforming move from the current solution, then the Steepest Ascent variation is guaranteed to arrive
124 Metaheuristic Concepts Chapter 5
at the global optimum within a single iteration. However, this iteration involves explicitly enumerating every solution, so it is usually not practical.
This basic iterative improvement procedure forms the core of every local search metaheuristic. They often add layers of sophistication in order to escape from local optima and continue the search, to diversify the search into other more promising areas of the search space, or to intensify the search around a promising area of the search space.
5.2
Iterated Search
Iterated Search procedures repeatedly apply a simple local search mechanism, choosing a different starting solution each time.
5.2.1
Repeated Local Search
Repeated Local Search is perhaps the simplest "metaheuristic", with the ability to escape from local optima and continue with the search. This is achieved by selecting a new initial solution and repeating the ascent, to a new local optimum, until some time limit or iteration count is reached.
If a population of solutions is available, then the simplest expression of repeated local search is random restart local search; simply randomly sampling another solution from the population. The ability to do this will depend on the problem. For example, a solution to the travelling salesman problem simply consists of an ordered list of cities visited, and any permutation of the cities will be a feasible solution. In this case constructing a random solution is trivial. In contrast, a solution to the Chinese postman problem consists of an ordered list of the arcs of a graph to be traversed, and not all sequences of arcs form valid tours; consecutive arcs must be adjacent. In this case constructing a random solution is considerably harder.
Local search heuristics start from a solution s and output a local optimum w, that is better than s. Many experimental results from the literature confirm that the quality of the initial solution strongly influences the quality of the local optimum. Therefore much effort has gone into ways of choosing the next starting solution.
There are two broad approaches to the choice of a new initial solution. Those methods which use a constructive heuristic to construct a new solution are commonly called multi-start methods (random restart is the simplest case of this). Other methods, known as iterative local search perform an operation to select a new initial solution by perturbing the local optimum reached by the previous iteration, in such a way that the search may continue.
5.2.1.1
Termination criteria
One design decision that is common to all the iterated search techniques is when to stop. In fact the need to include some method to terminate the heuristic is common to all metaheuristics that do not “get stuck” in local optima. Possible options include the following:
5.2 Iterated Search 125
• A time limit on the computation. This is a natural termination condition, since a consideration of run-time is one of the motivating factors for using heuristics at all over algorithms (in the worst case explicit enumeration). A time limit also provides a way of balancing the number of iterations with the speed of the iterations, since either of these measures alone can be misleading as to the effort required. Time limits will, of course, depend on the speed of the machine running the heuristic, and the efficiency of the implementation.
• A limit on the number of iterations. The advantages of an iteration limit are that it allows comparison that is independent of the hardware or software implementation used to run the heuristic.
• A time or iteration limit after some significant event, for example the last improvement in the best solution.
• A local optimum is reached.
5.2.2
Iterated local search
Iterated local search (ILS) is a conceptually simple metaheuristic that iteratively applies local search to perturbations of successive local minima. The reasoning is that if the current solution is a local minimum then there are likely to be some portions of it that are optimized, and hopefully we will be able to keep these portions and continue to optimize the others, after escaping from the local minimum. From a search space perspective, we attempt to remain in a close region of the search space, to explore it further, rather than moving to a completely different region.
Lourenço et al. [176] describe four procedures that have to be implemented in an ILS heuristic: GenerateInitialSolution, LocalSearch, Perturbation, and AcceptanceCriterion, and define the following high-level architecture, which is widely reproduced within the literature:
Algorithm 5.1 metaheuristic ITERATED LOCAL SEARCH
s0 = GenerateInitialSolution s* = LocalSearch(s0) repeat s′ = Perturbation(s*, history) s*′ = LocalSearch(s′) s* = AcceptanceCriterion(s*, s*′, history)
until termination condition met end
Within this basic framework, heuristics can either be very simple, or quite complicated. Lourenço et al. [176] give some tips for creating a basic version of ILS that usually leads to much better performance than random restart approaches: “(i) one can start with a random solution or one returned by some greedy construction heuristic; (ii) for most problems a local search algorithm is readily available; (iii) for the perturbation, and random move in a neighbourhood of higher order than the one used by the local search algorithm can be surprisingly effective; and (iv) a reasonable first guess for the acceptance
126 Metaheuristic Concepts Chapter 5
criterion is to force the cost to decrease, corresponding to a first-improvement descent in the set S*.” They consider the ILS approach to be a walk in the set of local optima, S*.
5.2.2.1
Initial solution
The usual method to obtain an initial starting solution is to apply some constructive heuristic, perhaps a greedy method. A recent (2008) example of using a constructive heuristic can be found in Tang and Wang [233], who extend the Insert/Delete heuristic of Mittenthal and Noon [193] for the travelling salesman subset-tour problem with one additional constraint to the Capacitated Prize-Collecting Travelling Salesman Problem.
If there are no well-regarded constructive heuristics for a given problem domain then any random solution can suffice; Stützle [232] uses a random assignment of items to locations for the quadratic assignment problem, stating that no high performing construction heuristics are known. Even in recent research the random initial solution is chosen, an example from 2009 is Grosso et al. [129], who apply ILS to finding maximin latin hypercube designs1, although they do give evidence that their approach could be improved by using a specialized heuristic to find the initial solution. Indeed, the premise of ILS is that starting each iteration from an already-promising solution is preferable to “any old solution”, so this would seem to apply to the initial starting solution also.
Dong et al. [72] develop an ILS for the permutation flowshop problem with total flowtime criterion, and conduct an experimental analysis of the effect of different initial solution generation methods on 120 benchmark instances and 900 randomly generated instances. They used a number of different constructive heuristics from the literature, and also randomly generated instances. Figure 5.1 and Figure 5.2 are reproduced with permission from [72] and demonstrate that the constructive heuristics all performed much better than the randomly generated solutions, based on the average relative percentage deviation (ARPD) between the method and best solution known:
1
The maximin LHD problem calls for arranging N points in a k-dimensional grid so that no pair of points share a coordinate and the distance of the closest pair of points is as large as possible.
5.2 Iterated Search 127
Figure 5.1: Performance of initial methods on benchmark instances [72]
Figure 5.2: Performance of initial methods on random instances [72]
5.2.2.2
Local search component
The basic form of the local search component is simply ascent search, although there can be considerable differences in the performance of different neighbourhood schemes, and even hybrids with other metaheuristics. Quite sophisticated neighbourhoods can be constructed using expert knowledge of the problem domain. However, even quite basic neighbourhood schemes can provide good results. Tang and Wang [233] utilize a sophisticated local search component which actually has three different search schemes. Every five iterations they run a basic form of the Tabu Search metaheuristic, which doesn’t technically end in a local optimum, since Tabu Search just takes the best non-restricted move available, even if it worsens the objective value, so they let it run for 80 moves. Every 20 iterations they find a local optimum using a 3-opt neighbourhood, and the remainder of the time they use a fast 2- opt neighbourhood called dynasearch. The 3-opt neighbourhood is more computationally intensive, which is why it is only performed infrequently. This multi-phase approach is an interesting hybrid of the iterated local search, Variable Neighbourhood Search and Tabu Search heuristics.
5.2.2.3
Perturbation
The perturbation step transforms the current solution s into a new starting solution s′. It is used to escape from the local optimum resulting from the local search heuristic. A key consideration here is the extent to which the perturbation changes the current solution. If too great a change is made then there is a danger of losing the good features of the current solution, and moving into a completely different region of the search space. In this case the ILS heuristic may not have any advantage over a random restart method. However, if the perturbation is too small then the perturbed solution s′ may be in what
Lourenço et al. [176] call the same basin of attraction as s, i.e. they result in the same local optimum during the next local search phase. For illustration consider Figure 5.3. At the end of the local search phase we are sitting at the local maximum a. If the perturbation only moves the solution to b, then the next ascent search will end up at a again; a and b are said to be in the same basin of attraction. If,
128 Metaheuristic Concepts Chapter 5
however, the perturbation moves the solution to c, then the search will arrive at a new local maximum
d, which in this instance is better than a, but in general this may not be the case.
Figure 5.3: Pictorial representation of the perturbation step for iterated local search.
Lourenço et al. [176] point out that the local search should not be able to undo the perturbation, otherwise the same local optimum is likely to be repeated. They state that often a random move in a neighbourhood of higher order than the one used by the local search component can achieve good results.
Tang and Wang [233] utilize a feature called backtracking, which they borrow from Congram [55]. In their implementation of backtracking, whenever the current solution has not improved for a given number of successive iterations, it is pulled back to the best one found in history. This attempts to ensure that most of the search time is spent in promising regions of the search space. We can easily see that this concept can be extended to revisiting other promising regions of the search space; not necessarily the best solution found so far, but perhaps one of a set of elite solutions. This approach of the perturbation involving past solutions is explored in more detail in the discussion of Path Relinking. The perturbation step is also sometimes known as a kick. This kick is usually performed by making the perturbation a small change on one or a few solution components, often at random. Tang and Wang [233] introduce a guided kick, which is essentially the evaluation of a set of possible small moves – adding a customer, removing a customer or swapping a customer – and then choosing the best of these moves. If none is improving then they perform a multi-customer swap and then restore feasibility if necessary. So the perturbation can sometimes be thought of as simply a modification of the neighbourhood for a move.
Lourenço et al. [176] refer to the strength of a perturbation as the number of solution components that are modified, for example in the TSP, it is the number of edges that are changed in the tour. They experimentally analyse perturbation strengths and show that for some problems, an appropriate perturbation strength is very small and seems to be independent of the instance size. However, for other problems the best permutation size is strongly dependent on the particular instance. They suggest using
5.2 Iterated Search 129
Another approach described by Lourenço et al. [176] is more complex than simply making a move in a higher order neighbourhood. They describe a general procedure of subtly modifying the problem instance, for example via the parameters defining the various costs, then running the local search on this modified problem to obtain a new solution. This is the approach used by Baxter [18], which may be the earliest application of an iterated local search approach, although he simply called it local optima avoidance.
Dong et al. [72] also define a parameterized perturbation, which is a number of pair-wise swaps of the solution components. The number of swaps to perform is their perturbation strength. They perform an experimental analysis of various permutation strengths, and determine that the optimal level for their flowshop problem is between 4 and 7, based on the average relative percentage deviation (ARPD) between the method and best solution known, as shown below in Figure 5.4 and Figure 5.5, which are reproduced with permission from [72]:
Figure 5.4: Performance of perturbation strengths on benchmark instances [72]
Figure 5.5: Performance of perturbation strengths on random instances [72]
5.2.2.4
Acceptance Criterion
Iterated local search may be thought of as a walk in the space of local optima [176], with the neighbourhood of this space being defined by the local search components executed at each step and the perturbation mechanism. The AcceptanceCriterion procedure then determines whether that “move” in local optima space will occur, or whether the search will consider another locally optimal “neighbour” of the current solution. This procedure can be used to control the balance of intensification and diversification of the search process.
A strong intensification effect is obtained by requiring that the neighbouring local optimum be better than the current local optimum, this would be analogous to a first-improving ascent search. We couldn’t find an example of this in the literature, but it is easy to extend this analogy and consider a
best-improving ascent search in the space of local optima, which would correspond to making all the available perturbations, and applying the embedded local search component to each one, then selecting
130 Metaheuristic Concepts Chapter 5
the best “neighbouring” local optimum. This type of search would be very computationally intensive, however.
Lourenço et al. [176] summarised a finding of Stützle [231], who analysed the run-time behaviour of ILS heuristics for the TSP, and concluded that the “better” acceptance criterion led to a type of stagnation behaviour for long run times, as expected for a strong intensification search.
To obtain a strong diversification effect, the neighbouring local optimum could always be accepted, regardless of whether it is better or worse than the current solution. The acceptance criterion in ILS is equivalent to the admissibility criteria in the MLS framework, and therefore any of the methods discussed in later sections for using these criteria are possible. For example, in Martin et al. [184,185] a hybrid of ILS and Simulated Annealing is used, where the acceptance criterion is based on the Simulated Annealing idea: s*′ is always accepted if it is better than s*, otherwise if is accepted with probability exp{C(s*) – C(s*′)) / T}, where C is the objective function value (based on a minimization
objective) and T is a parameter called the annealing temperature.
Lourenço et al. [176], in their review of ILS for the 2003 Handbook of Metaheuristics, state that: “Most of the acceptance criteria applied so far in ILS algorithms are either fully Markovian or make use of the search history in a very limited way. We expect that there will be many more ILS applications in the future making strong use of the search history; in particular, alternating between intensification and diversification is likely to be an essential feature in these applications.”
This has been true in that ILS perturbations are incorporated as key aspect of many modern hybrids that arise. A general trend seems to be that “pure” forms of the various metaheuristic families are starting to be become less prevalent in the literature; they are being replaced by sophisticated hybrids that take the features of many metaheuristics and mix them into adaptive, multi-phase approaches.
5.2.2.5
MLS interpretation
As with many metaheuristic concepts, Iterated Local Search may be interpreted in more than one way. The most natural interpretation is to consider a default MLS search scheme as the ILS LocalSearch component. We would then implement the perturbation using MLS triggers and responses. After the
local optimum trigger is tripped, a perturbation response module is performed. This occurs within the MLS control system, leaving the search scheme as is. The perturbation response module acts as a ChangeCurrentSolution module, using whatever perturbation logic is specified by the heuristic. This module would also include the acceptance criterion logic to determine whether to accept the perturbed solution or not.
An alternative interpretation is to follow Lourenço et al. [176] and consider the ILS heuristic as a walk in local optimum space. In this case the search scheme would be quite non-standard, but there wouldn’t