Since the set of solutions of a COP is a finite set, a naive proposition to solve it would be to do an exhaustive enumeration of this set and select the best solution. However, this set is frequently so large that complete enumeration of all possible solutions is not doable even in the fastest computers available. An example with the simplest problem explained—the TSP—gives a dimension to the former statement. The size of the search space in this problem is n! possible solutions. Given different values of n, Table 2.5 shows the combinatorial explosion of the number of solutions as the number of cities grows. Consequently, we need to turn to smarter methods.
The COP explained in Section 2.2 are NP-hard. This is to say that, no polyno-mial time algorithm has been designed, or is expected to be designed, to solve them to optimality. These problems are also referred as being computationally intractable.
Depending on their size, two approaches can be considered to search their solution space and solve them: exact and approximate, refer to Fig 2.2. Nevertheless, a solution methodology may combine both strategies at different stages. This thesis presents a methodology of this type.
2.3.1 Exact Methods
These procedures, also known as complete methods, explore the search space exhaus-tively, but not by merely enumerating all the solutions. A common technique to im-plement this idea is to keep an incumbent solution, and try to prove that specific regions of the search space cannot contain better solutions, and as a consequence, can be avoided (pruned). However, if the search strategy is unfortunate for the problem
2.3. Overview of Solution Methods 19
Table 2.5: Effect of the size of the instance on the number of feasible solutions.
Number of cities, n Size of the search space
5 120
10 3,628,800
75 2.5 ×10109
100 9.3 ×10157
Optimisation Methods
Exact methods
Dynamic programming
Constraint programming
Branch & X
Branch & bound
Branch & cut
Branch & price
Approximate methods
Heuristic algorithms
Metaheuristics
Single-solution based
Population-solution based
Problem-specific heuristics
Approximation algorithms
Figure 2.2: Classical optimisation methods for COP.
being solved, an exhaustive or almost exhaustive search can occur. A very important advantage of exact methods is that they guarantee that the solution found is optimal if the procedure is given sufficient time and space.
The worst-case running time of an exact method for large-sized instances (or even medium-sized) of many NP-hard problems is still very poor (exponential). They may take numerous hours to find moderately decent, let alone optimal, solutions. However, the size of the instance is not the unique indicator to describe the difficulty of the problem, but also its structure. It is possible that for a given problem some small in-stances are very hard to solve to optimality while larger inin-stances may be solved exactly more easily. Since most vehicle routing problems are computationally intractable, the current limit of exact algorithms is a network of around one hundred customers. Ap-proximate approaches are applied when solving the much larger instances met in many
industries. The following paragraphs explain the classical exact algorithms: dynamic programming, the branch & X family and constraint programming.
Dynamic Programming. The idea underlying this technique is to represent a prob-lem by a decision process (minimisation or maximisation) which proceeds in a series of stages. Dynamic programming (DP) decomposes a problem into a number of smaller subproblems (also known as states) each of which is further decomposed until subprob-lems have trivial solutions. In this decomposition process, partial results are stored, and later, whenever they are needed, they are looked up instead of recomputed. Thus, it gains efficiency by avoiding solving common subproblems many times. The term “dy-namic programming” stems from control theory. Programming means that an array or tableau is used to store partial results which lead to the construction of a solution.
Therefore, it is a bottom-up approach. The procedure avoids total enumeration of the search space by pruning partial results that cannot lead to the optimal solution. Al-though it may seem that any optimisation problem can be solved using DP, this is not the case. The optimisation problem must exhibit two properties:
(1) Bellman’s principle of optimality—an optimal solution to an instance of a prob-lem always contains optimal solutions to all subinstances—must apply in the problem (Bellman, 1957).
(2) The space of subproblems should be relatively small. The decomposition done by this technique seems to lead to an exponential-time algorithm, which is indeed true in the TSP (Neapolitan and Naimipour, 1998). Typically, however, the total number of distinct subproblems is polynomial in the input size.
For instance, the Shortest Path Problem complies with Bellman’s principle, and DP has become the standard method to solve it. In VRP, DP is usually utilized together with other solution methods, for example metaheuristics, in order to solve some specific portion of the problem.
Branch & X Family. These procedures are based on an implicit enumeration of all the solutions of the problem considered. A branch & X algorithm uses a divide and conquer strategy to partition the solution space into subproblems and then optimises each subproblem individually. The search space is explored by dynamically building a tree whose root node represents the problem being solved and its whole associated search space. The leaf nodes are the potential solutions and the internal nodes are subproblems of the total solution space. In this tree, the solution set of the child nodes is equal to the solution set of the parent nodes. In branch-and-bound (B&B), the pruning of the search tree is based on a bounding function that prunes subtrees that do not contain any optimal solution. However, the efficiency of the method relies upon its ability to prune nodes early in the tree, before they produce too many child nodes. A branch-and-cut method is similar, but valid inequalities are added to the mathematical description of the problem to further prune non-promising areas of the
2.3. Overview of Solution Methods 21
search space. When B&B considers column generation methods, a branch-and-price scheme appears.
These procedures, commonly referred as integer linear programming, have been used extensively in the solution of vehicle routing problems, but with the limitation that they can solve efficiently only small or medium-sized instances. Toth and Vigo (2014) provide a good introduction to the application of these methods on the CVRP, while Nemhauser and Wolsey (1999) provide an in-depth explanation of these methodologies applied to different combinatorial optimisation problems.
Constraint Programming. Constraint Programming (CP) (Rossi et al., 2006) is a paradigm for representing and solving a wide variety of problems. CP combines techniques from artificial intelligence, logic programming, and operations research.
CP is traditionally an exact approach. However, by virtue of the decoupling between modelling of the problem and searching for a solution that it is capable of doing, many alternative search methods, not necessarily complete, have been devised.
Problems are modeled in terms of decision variables, domains for those variables and constraints between the variables. In addition to expressions involving the usual arithmetic and logical operators, complex symbolic constraints can be used to describe a problem. The problems are then solved using complete search techniques such as depth-first search (for satisfaction) and B&B (for optimisation). However, for routing problems of practical size, complete search methods cannot produce solutions in a short time. By contrast, iterative improvement methods (refer to 2.4.2) have proved very successful in this regard. Iterative improvement methods operate by changing small parts of the solution, for instance moving a visit from one route to another.
CP enhances the search using constraint propagation. If bounds or constraints on a variable can be inferred, or are tentatively set, these changes are “propagated” through all the constraints to reduce the domains of constrained variables.
CP has proved to be a very efficient approach for solving job scheduling problems, but such is not the case for VRP. Very few efficient methods based on CP have been published for solving routing problems. Shaw (1998) coupled a local search method he termed Large Neighborhood Search (LNS) with constraint-based technology to solve vehicle routing problems. LNS explores a large neighborhood of the current solution by removing customer visits from the set of planned routes, and re-inserting these visits using a constraint-based tree search. His results over benchmark problems showed his method to be competitive with OR metaheuristic procedures. De Backer et al. (2000) introduced a CP model for vehicle routing, and a system for integrating CP and it-erative improvement techniques. They coupled their itit-erative improvement technique with a metaheuristic to avoid the search getting trapped in local minima. Two meta-heuristics were investigated: a simple tabu search procedure and guided local search.
An empirical study over benchmark problems shows the relative merits of their tech-niques.
2.3.2 Approximate Methods
These procedures represent a possibility to overcome the performance issue of exact methodologies. Approximate methods, also known as incomplete, explore the search space in a non-exhaustive (even, possibly, stochastic) way usually improving an in-cumbent solution iteratively. Many approximate methods cannot offer any guarantee about the quality of the solutions found, because they cannot detect when the search space has been completely explored, which might never happen. On the other hand, they have two important advantages. First, since they do not have to be exhaustive, they can explore promising regions of the search space much earlier than exact meth-ods. Second, since they can move freely through the search space, without sticking to a fixed systematic exploration rule, they usually exhibit better anytime properties, i.e., the more time is given to the search method, the better is the returned solution.
Two kinds of algorithms compose this class: approximation algorithms and heuristic algorithms.
Approximation Algorithms. Unlike heuristics, which usually find good quality solutions in a reasonable time but without guarantees, approximation algorithms pro-vide provable solution quality and provable run-time bounds. For example, an -method guarantees that the solution obtained is at most times more costly than the best so-lution attainable. The aim in designing an approximation algorithm for a problem is to find tight worst-case bounds. It gives more knowledge on the difficulty of the prob-lem and can help to design efficient heuristics. However, this type of algorithms are problem-dependent and this limits their applicability. Moreover, for real-life applica-tions it might not be possible to design a polynomial-time approximation algorithm with constant error guarantee or approximation can be impractical. This is to say, error guarantee may be too poor or the running time of the algorithm may be too high. For example, when solving the TSP using benchmark instances, the best approximation ratio is 1.5, whereas the best heuristics are less than 1% away from optimality. Vazirani (2003) provides an in-depth presentation of these algorithms.
Heuristics. The concept of heuristic search as an aid to problem solving was firstly introduced by Polya (1945). A heuristic is a solution method which seeks (and hopefully finds) a feasible solution of good quality within modest computing time. Modest is defined more exactly as a time which is bounded by a low-order polynomial function.
A heuristic is approximate in the sense that it provides (hopefully) a good solution for relatively little effort, but it does not guarantee optimality. Neither does it provide an approximation guarantee on the obtained solutions. These procedures are tested empirically and the quality of the solutions obtained can be evaluated by comparing them against the output of an exact method when possible, or against the output of other heuristic approach. This implies the existence of standard data sets. Heuristics are conspicuously preferable in practical applications because of their easiness to adapt them to diverse and complex constraints (usually present in real problems), and their ability to solve large instances in polynomial time. These techniques typically require an ad hoc construction of the algorithm applied.