Combinatorial Optimization Methodologies

Chapter 4 mathematically defined the SDDLBP and stated that it belongs to the class of NP- complete problems, necessitating specialized solution techniques. This dissertation makes use of techniques from combinatorial optimization which is a brand of optimization in applied mathematics and computer science that is related to operations research (OR), algorithm theory, artificial intelligence (AI) and computational complexity theory. Recent heuristic and metaheuristic innovations have tended frequently to become aligned with AI and OR. Hence in order to investigate what is current in heuristic or metaheuristic ideas, it is appropriate to examine procedures that have acquired some of the imprint of the AI and OR domain. In the domain of combinatorial optimization, the set of feasible solutions is discrete or can be reduced to one with the goal of finding the best possible solution. Solution methods including heuristics and metaheuristics generally provide near (optimal) solutions.

While exhaustive search works well enough in obtaining optimal solutions for small sized instances, its exponential time complexity limits its application on the large sized instances. Combinatorial optimization techniques allow for the generation of a solution which may be optimal or sub-optimal. In this dissertation, seven techniques are proposed making use of well- known metaheuristics for study and comparison. These include ant colony optimization (ACO), river formation dynamics (RFD), tabu search (TS), particle swarm optimization (PSO), simulated annealing (SA), hybrid genetic algorithms (HGA) and artificial bee colony (ABC) metaheuristics. These processed are then applied to the two instances from Chapter 5 for analysis, evaluation and comparison.

Exhaustive search (Brute-force search) requires the checking of every possible solution in order to determine the optimal solution. The drawback is that the most spaces are extremely large and an exhaustive search of combinatorial optimization problem instance will only take a reasonable amount of time for small sized examples. However, exhaustive search techniques will fail to find a solution for any instance except small sized instances within any practical length of time. Therefore, intelligent search techniques are required to solve such problems.

A heuristic is any problem-specific step by step set of procedures or rules. In this dissertation, the following task assignment rules are used as heuristics integrated to the proposed methods: Greatest Ranked Positional Weight (GRPW), Longest Processing Time (LPT), Shortest Processing Time (SPT), Greatest Number of Immediate Successors (GNIS), Greatest Number Of Successors (GNS), Random Priority (RP), Smallest Task Number (STN), Greatest Average Ranked Positional Weight (GARPW), Smallest Upper Bound (SUB), Smallest Upper Bound Divided by the Number of Successors (SUBNUS), Greatest Processing Time Divided by the Upper Bound (GPTUB), Smallest Lower Bound (SLB), Minimum Slack (MS), Minimum Number of Successors Divided by Task Slack (MNSTS), Greatest Number of Immediate Predecessors (GNIP).

A local search is a type of heuristic often used in solving combinatorial optimization problems. Given a search space, local search starts from an initial solution and then iteratively moves from one possible solution to another in search of a solution that is both feasible and better performing than the best found so far. This continues until some stopping condition is met. In this dissertation, some local search techniques are used as embedded to the proposed methods. A hybrid makes use of two or more solution generating techniques in sequence or repeatedly

during each iteration. This dissertation makes use of hybrids that combines local search with genetic algorithms.

A metaheuristic is a general purpose heuristic that provides a top-level strategy to guide other heuristics in finding feasible solutions within the search space. In practice, they are often seen having a stochastic component. Metaheuristics may include iterative, deterministic or stochastic search methods such as evolutionary algorithms and swarm intelligence techniques.

ACO first published by Dorigo et al. (1996) is a swarm intelligence based global search method for difficult combinatorial optimization problems like Travelling Salesman Problem (TSP) and Quadratic Assignment Problem (QAP). Inspired by colonies of ants, the ACO metaheuristic makes use of computer agents known as ants. A moving ant lays some pheromone in varying quantities on the ground, thus marking the path by a trail of this substance. While an isolated ant moves essentially at random, an ant encountering a previously laid trail can detect it and decide with high probability to follow it, thus reinforcing the trail with its own pheromone. The more ants follow a trail, the more attractive that trail becomes for being followed. The process is thus characterized by a positive feedback loop, where the probability with which an ant chooses a path increases with the number of ants that previously chose the same path. In brief, they consist in copying the method used by ants to find good paths from the colony to food sources.

RFD first published by Rabanal et al. (2007) is also a swarm based global search method for solving combinatorial optimization problems. In RFD method, instead of associating pheromone values to edges as in ACO method, altitude values are associated to nodes. Drops (represents ants in ACO method) reduce or increase the altitude of nodes as they move. The

probability of the drop to take a given edge instead of others is proportional to the gradient of the down slope in the edge, which in turn depends on the difference of altitudes between both nodes and the cost of the edge. Drops are unleashed at the origin node until they fall in the destination node. New drops are inserted in the origin node to transform new paths. After some steps, good paths from the origin to the destination are found. These paths are given in the form of sequences of decreasing edges from the origin to the destination.

TS first introduced by Glover (1989, 1990b) is a metaheuristic strategy to overcome local optimality in solving combinatorial optimization problems. The underlying idea is to forbid some search directions at a present iteration in order to avoid cycling, but to be able to escape from a local optimal point. This strategy can make use of any local improvement technique. The major theme behind TS is to incorporate flexible memory functions into the search procedure. TS is distinct from the SA and GA methods in that SA and GA in terms of memory such that SA and GA are memoryless and probabilistic random search methods, while TS is deterministic and takes advantage of the history of the search process. There are many problems that are successfully solved using tabu search.

PSO first introduced by Kennedy and Eberhart (1995) is derived from the social behaviour of migrant birds and fish which migrate in flocks and schools, respectively, while sharing the information among themselves to locate their foods. In the PSO method, each member is called a particle. Each particle has a position and moves around in the multi- dimensional space with a velocity. To find the optimal solution, each particle adjusts its flying by constantly updating its velocity according to its own flying experience and its companions’ flying experience. A particle flies in the problem search space towards the optimal position.

SA initially proposed by Kirkpatrick et al. (1983) is a stochastic neighbourhood search method that is based on the analogy between the process of annealing of solids and the solution methodology of combinatorial optimization problems. Analogous to its use with the physical annealing of solids, the combinatorial optimization problem solution undergoes a series of changes while looking for an improved solution according to some objective function. As simulated annealing starts, an initial solution is generated and used as the current solution. If the objective function value is superior to that of the current solution, the neighbouring solution becomes the new current solution. If the neighbouring solution provides an objective function value inferior to that of the current solution, the neighbouring solution may still become the current solution if a certain acceptance criterion is met. As the temperature drops, new neighbouring solutions to the current solution are found. SA technique has the capability of jumping out of the local optima for global optimization which is achieved by accepting with probability neighbouring solutions worse than the current solution. The acceptance probability is determined by a control parameter (T ) which decreases during the SA procedure according to a g

cooling rate. The process of finding neighbouring solutions and accepting these as current solutions if acceptance criteria are met is repeated according to the cooling pattern until some stopping criteria is met.

GA first proposed by Holland (1975) is an adaptive search method for solving complex optimization problems. Goldberg (1989) presented a number of applications of GA to search, optimization and machine learning problems. In general, the power of GA comes from the fact that the technique is robust, and can deal with a wide range of problem areas. A GA emphasises genetic encoding of potential solutions into chromosomes and applies genetic operators such as crossover and mutation to these chromosomes. Each individual represents a potential solution to

the problem at hand, and is evaluated to give some measure of its fitness. Some individuals undergo stochastic transformation by means of genetic operations to form new individuals. After several generations, the algorithm converges to the best individual, which hopefully represents an optimal or suboptimal solution to the problem (Gen and Cheng 2000).

Inspired by the intelligent foraging behaviours of honeybee swarms, Karaboga (2005) proposed ABC algorithm that implemented a new swarm intelligence based optimization technique. It classifies foraging artificial bees into three groups as follows: employed bees, onlookers, and scouts. An employed bee is responsible for flying to a food source and collecting from the food that is exploited by the swarm. An onlooker decides on whether a food source is acceptable or not according to dances performed by the employed bees. A scout randomly searches for exploration of new food sources. In the ABC algorithm, each solution to the problem under consideration is called a food source and represented by an n dimensional vector where the fitness of the solution corresponds to the nectar amount of the associated food resource. As with other intelligent swarm-based approaches, the ABC algorithm is an iterative process. There are few control parameters in the ABC algorithm, which is the main advantage of the algorithm. Due to its simplicity and ease of implementation, the ABC algorithm has gained more attention and has been used to solve many practical engineering problems.