Summary

In this chapter, we have given introduction to the types of heuristics that we will use as elementary components in the studies described in the following chapters. We also reviewed some research areas related to this thesis and gave comparisons between them and our intended studies. In the following chapter, we begin with a basic architecture that serves as the foundation of our study on neighborhood-based heuristics. Then in Chapter 6, we will switch to the topic of combining multiple sampling-based heuristics.

Chapter 3

An Architecture for Combining Neighborhood-based Heuristics

In this chapter, we present a basic architecture that will serve as the underlying foundation for the first part of this thesis. It functions by passing the work of one heuristic to another in the form of a complete solution. This complete solution will be used by the receiving heuristic for initializing its own operation. As mentioned in the previous chapter, the class of heuristics that we consider in the first part of this thesis are neighborhood-based heuristics. One characteristic of the neighborhood-based heuristics is that they can pick up a complete solution and modify it to generate a new solution. Thus, we can design a high-level flow that chains the effort of multiple heuristics in a pipelined fashion. In this way, we can create an environment that allows heuristics to collaborate with each other.

Another view on this idea is that we are essentially structuring the exploration of the search space as a process that constructs many solution chains. Each solution chain is formed by successively applying one of a set of heuristics (chosen by certain policy) to the current solution to generate the next solution. The algorithmic description of this idea is shown in Algorithm 6. In this algorithm, we assume that we are given a set of heuristics H, a policy L for choosing the chain length, and a policy H for choosing among the given heuristics. Every iteration starts with picking a number ` based on the policy L. This ` will bound the length of the next solution chain. The algorithm then goes on constructing a chain of solutions by applying a sequence of heuristics (selected according to policy H) successively, as illustrated in Figure 3.1. If any solution encountered during this process is

Algorithm 6 A Basic Architecture for Heuristic Collaboration

Require: a set of heuristics H, a policy L for choosing lengths, and a policy H for choosing heuristics.

1: x ← initial solution.

2: while stopping criteria not met do

3: ` ← a length chosen according to L

4: x₀ ← x

5: for i = 0 to (` − 1) do

6: h_i+1 ← a heuristic chosen from H according to H.

7: x_i+1← h_i+1(x_i)

better than the incumbent solution x, we break out of the inner loop and replace x with the better solution. Otherwise, this process repeats. Note that since in most cases, we have stochastic heuristics included in H, we can also think of this process as sampling a solution chain.

For this research, we mainly consider the situation where the set of heuristics H is predefined and is given to us from some external source. Thus, we can think of a problem domain as a combination of an optimization problem and a set of heuristics designed for that problem. Apparently, there are still two elements that we need to supply to make Algorithm 6 functional: the policy L for choosing the chain lengths and the policy H for choosing the heuristics. For this preliminary study, we will mainly focus on the policy L.

Specifically, we will center our discussion on a setting that makes two agnostic assumptions:

1. We assume that we do not have detailed knowledge about the problem domain being solved, except that we have access to the objective function (treating it as a black-box) and a set of predefined heuristics.

2. We assume that we have no information about the amount of time allocated for running our algorithm.

We develop our discussion based on these two assumptions and will come back to the

x₀ x₁

Check if xi is better than incumbent solution x.

If it is, we terminate this chain at xi.

Figure 3.1: Sampling a solution chain. This process starts with a solution x₀. Each following step consists of selecting a heuristic hi+1 from the set of provided heuristics H and applying h_i+1 to the previous solution x_i to get a new solution x_i+1. If x_i+1 is better than the incumbent solution x, we terminate this process and replace x with x_i+1. Otherwise, we proceed until reaching the bound `. Since in most cases, we have stochastic heuristics included in H, and hence the x_i’s are generated stochastically, we can think of this process as sampling a solution chain.

issue of designing H in the following chapters. For simplicity, the following discussion will use a policy H_u that makes a uniformly random choice among the provided heuristics H each time it is consulted. We should emphasize that this particular policy is for illustrative purpose. The techniques presented in the following generalizes to arbitrary H. And in the next chapter, we will start exploring mechanisms that automatically derive H policies from data.

Conceptually, we can see that this architecture is an algorithmic template that we can plug different policies into to make it behave differently. It also relates to a growing research field called hyper-heuristics. In the following section, we will start by briefly reviewing the field of hyper-heuristics and connecting our algorithm to this field.

3.1 Hyper-heuristics

The term “hyper-heuristics” [14, 16, 21] refers to a set of methods that aim at achieving the following objective: given a set of heuristics (or building blocks that can be used to construct heuristics), automatically produce an adequate combination that handles the given problem. This framework has a two-layer structure: At the lower level, there is a set of heuristics or building blocks for constructing heuristics, and the top level

corre-sponds to a mechanism that utilizes the lower-level components in combination to solve a given problem. The term “hyper-heuristics” was first introduced by Cowling et al. [26] as

“heuristics to choose heuristics.” Its definition has since been extended. Broadly speak-ing, hyper-heuristic approaches can be roughly categorized into two classes: methods that harness a pre-existing set of heuristics, and methods that generate new heuristics. In the first category, the hyper-heuristics are equipped with a set of predefined low-level heuris-tics, and the task is to decide which low-level heuristic to apply at a given point during the optimization. The second category corresponds to methods that build new heuristics from a set of components, usually via genetic programming (e.g. [11], [51], [58].) For our purposes here, we focus on the connection to the first category.

Most of the methods from the first category follow an iterative procedure: Given an initial solution (either generated randomly or heuristically), the hyper-heuristic loops through the steps of (i) selecting a heuristic from the set of provided heuristics (usually neighborhood-based heuristics), then (ii) applying the selected heuristic to the incumbent solution to generate a new solution, and finally (iii) deciding whether the new solution should be accepted as the new incumbent solution. This process iterates until the termi-nation criteria are met.

Another dimension for classifying hyper-heuristics is based on the learning mechanisms they employ. A common classification is to categorize them as online learning, offline learning or no learning at all. An online learning hyper-heuristic adjusts itself based on the feedback received during the search process and dynamically biases the selection prob-abilities (e.g. [75], [72], [61]). Offline learning, in contrast, takes place before the actual search starts (e.g. [71], [4]). While most of the recent hyper-heuristics employ some forms of learning, in this chapter, we will focus on the no-learning cases because it simplifies our discussion. Furthermore, as we will see in Section 3.4, a no-learning algorithm can still be pretty competitive when comparing to more elaborated, learning-based hyper-heuristics.

Hyper-heuristics without a learning mechanism were among the first hyper-heuristics to appear. In their initial paper, Cowling et al. [26] proposed a no-learning hyper-heuristic called Simple Random which chooses a low-level heuristic uniformly randomly at each step.

The authors experimented with two acceptance strategies: All Moves (AM) and Only Improving (OI). AM accepts new solutions regardless of their quality, while OI accepts

only better solutions, and if the new solution is inferior than the incumbent, it is simply discarded.

Now consider the algorithm that was previously presented as Algorithm 6. If we use a simple policy Hu that selects a heuristic uniformly at random each time it is consulted, then it can be seen as an extension to the Simple Random with OI. And instead of a single-step heuristic application, in Algorithm 6 we consider the accumulated effect of applying multiple heuristics. More specifically, we consider the exploration of solutions within a chain of limited length. As described before, this solution chain is formed by successively applying a randomly chosen heuristic to the previous solution to generate the next solution.

For this setup, the only unspecified element of the algorithm is how to make choices on the bounding length `, and the algorithm can be viewed as a procedure which at each iteration of the inner loop, samples a solution that is at most ` operations (i.e. heuristic applications) away from the incumbent solution x. In the following, we will often use the word “operation” to refer to the action of applying a heuristic.

In this chapter, we discuss policies for choosing `. One can imagine that this choice can have a significant impact on the algorithm. For example, if the nearest improving solution is k operations away, then choosing an ` < k will not yield any improvement. On the other hand, if we choose an ` that is much larger than needed, we can waste a significant amount of time exploring unpromising regions. Furthermore, proper choice of ` may vary during the optimization process, possibly making rigid policies (e.g. always choose ` = 10) inefficient or ineffective.

In the following section, we first define the scenario that we consider, with its two ag-nostic assumptions. Based on that, Section 3.3 presents a strategy for choosing ` which has an asymptotic guarantee. Following that, Section 3.4 shows the result of our preliminary experiments. We offer some further discussion of the results in Section 3.5. And finally, Section 3.6 summarizes this chapter.