Although exact approaches provide an optimal solution, they become impractical even on graphs with several hundreds of vertices. Therefore, when one deals with the MIS problem on very large graphs, in which the exact approaches cannot be applied, heuris- tics provide a possible option.
2.3.1 Greedy Heuristics
Sequential Greedy
The majority of approximation algorithms in the literature for the MIS problem are called sequential greedy heuristics. These heuristics generate a maximal independent set through repeated addition of a vertex into an independent set or repeated deletion of a vertex from the original graph. Borowiecki et al. [25] called the two strategies best-in and worst-out strategies, respectively. Decisions on which vertex to be added in or moved out next are based on certain indicators associated with candidate vertices. For example, a possible best-in heuristic constructs a maximal independent set by repeatedly adding in a vertex that has the smallest degree among candidate vertices. In this case, the indicator is the degree of a vertex. On the other hand, a possible worst-out heuristic can start with the whole vertex set V and then repeatedly remove a vertex out of V until V becomes independent. Three well known heuristic algorithms
2.3 Heuristic Methods 21
are Vertex Order (VO) [132], MIN [145], and MAX [83]. Algorithm MAX follows worst- out strategy using degree indicator while MIN and VO follow best-in strategy with the same indicator. Moreover, while MIN and MAX update the indicators every time when a vertex is added in or moved out, we call this approach as new strategy, while VO does not, but follows so-called old strategy. All three algorithms give a maximal independent set in polynomial time. However, under some restrictions, these maximal independent sets become maximum. Borowiecki et al. [25] suggested a more general indicator, so- called potential function for greedy algorithms. We describe these methods more detail in Chapter 6.
Local Search Heuristic
A common feature of the sequential heuristics is that they all find only one maximal independent set. Once a maximal independent set is found, the search stops, hoping it is (close to) the optimal solution. This suggests us a possible way to improve our approximation solutions by expanding the search. For example, once we find a solution S, we can search its ’neighbors’ to improve S. This leads to the class of the local search heuristics. It is worth to notice that this improvement technique also leads to so-called augmenting methods, which are described more in Section 2.5 and Chapter 3.
Greedy Randomized Adaptive Heuristic
A class of heuristics designed to search random various neighbors of some maximal solution S is called the randomized heuristics. A greedy randomized adaptive search procedure (GRASP) is an iterative randomized sampling technique, in which, each iteration provides an heuristic solution to the problem at hand. The best solution over all GRASP iterations is kept as the final result. An elaborated implementation of the randomized heuristic for the MIS problem was described by Feo et al. [62].
Continuous-based Heuristics
Recently, continuous formulations of discrete optimization problems turn out to be particularly attractive. They not only allow us to exploit the full arsenal of continuous techniques, thereby leading to the development of new algorithms, but may also reveal unexpected theoretical properties. In 2002, Burer et al. [40] derived two continuous optimization formulations for the MIS problem. Based on these formulations, they developed and tested new heuristics for finding large independent sets. In the same year, Busygin et al. [41] proposed a heuristic for the MIS problem which utilizes classical results for the problem of optimization of a quadratic function over a sphere.
2.3.2 Advanced Search Heuristics
Local search algorithms are only capable of finding local solutions of an optimization problem. In the past few years, many powerful variations of the basic local search procedure have been developed and applied in the MIS problem to avoid this problem. Many of which are inspired from various natural phenomena, which we describe briefly in this subsection.
22 2 Techniques for Finding Maximum Independent Sets
Simulated Annealing
In condensed-matter physics, the term "annealing" refers to a physical process to obtain a pure lattice structure, where a solid is first heated up in a heat bath until it melts, and next cooled down slowly until it solidifies into a low-energy state. During the process, the free energy of the system is minimized, which we suppose that it coresponds to the optimal solution of the problem. Simulated annealing was introduced in 1983 by Kirkpatrick et al. [108]. Here, the solutions of the problem correspond to the states of the physical system, and the evaluation value of a solution is equivalent to the energy of the state.
Aarts and Korst [1], without presenting any experimental result, suggested the use of simulated annealing for solving the MIS problem using a penalty function approach. Neural Networks
Artificial neural networks (or simply, neural networks) represent an attempt to imitate some of the useful properties of biological nervous systems, such as adaptive biological learning. A neural network consists of a large number of parallel, highly interconnected processing elements emulating neurons, which are tied together with weighted connec- tions analogous to synapses. In the mid-1980’s, Hopfield and Tanks [104] showed that certain feedback continuous neural models are capable of finding approximate solutions to difficult optimization problems. Aarts and Korst [1] provided an excellent introduc- tion to a particular class of neural networks (so-called the Boltzmann machine) for the MIS problem. Other examples about attempts at encoding the MIS problem of a neural network were given by Ballard et al. [15], Ramanujam and Sadayappan [152], and Takefuji et al. [160].
Genetic Algorithms
Genetic algorithms is an optimization method motivated by evolution processes in nat- ural systems. They work on a population of solutions which are called chromosomes or individuals. Each individual has an associated fitness value which determines its probability of survival in the next generation. The higher the fitness, the higher the probability of survival. The genetic algorithm starts out with an initial population of members generally chosen at random and makes use of three basic operators reproduc- tion, crossover, and mutation. Reproduction consists of choosing the chromosomes to be copied in the next generation according to a probability proportional to their fit- ness. The crossover operator is applied between pairs (or more) of selected individuals to produce new ofsprings having properties from their parents. The mutation operator is applied which randomly changes a chromosome. An introduction about genetic al- gorithms and some practical examples can be found in [135]. One of the first attempts to solve the MIS problem using genetic algorithm was done in Bäck and Khuri [14]. Hifi [102] also modified the basic genetic algorithm and applied it to the MIS problem. Tabu Search
Tabu search, introduced independently by Glover [78, 79] and Hansen and Jaumard [91], is a modified local search algorithm, in which, a prohibition (tabu) based strategy is employed to avoid cycles in the search trajectories and to explore new regions in