Parallel genetic algorithms

The major motivations behind parallel implementations of genetic algorithms (Stender 1993; Cantu Paz 1998; Cantu Paz 2000) include:

• For some problems, the populations in a GA need to be very large and the memory required to store each individual might be excessive. In some cases this makes it impossible to run an application efficiently using a single sequential implementation, hence some parallelisation form of GA is desirable.

• Fitness evaluation is usually very time-consuming. Parallel processing of fitness evaluations could greatly reduce computational time.

• Parallel implementations of GAs can search in parallel different subspaces of the hypothesis space, thus making it easier for the GA to escape from low-quality sub-regions of the search space.

Cantu Paz (1998) proposed four classifications of parallel GAs: global single-population master-slave GAs, single-single-population fine-grained GAs, multiple-population coarse-grained GAs, and hierarchical parallel GAs which are the combinations of the first three parallel GAs. Figure 2.12 below schematically describes the first three types of parallel GAs, adapted from Cantu Paz (1998).

82

(c)

(b) (a)

Slave Master

FIGURE 2.12 (a) master-slave parallel GA. (b) fine-grained parallel GA. (c) multiple-population parallel GA.

Figure 2.12(a) shows a master-slave parallel GA. The master machine stores the population to be evolved and executes GA operations. The master machine also distributes individuals to slave machines so that slave machines can evaluate the fitness of individuals in a parallel fashion. The slave machines only conduct fitness evaluations, not genetic operations such as crossover or mutation. Master-slave parallel GAs use global information, i.e. the entire population, for selection and genetic reproduction, because only a single population is saved in the master machine.

In this sense, master-slave parallel GAs are the same as the classic simple GAs. One of the practical issues concerning master-slave parallel GAs is the communication overhead between the master and slave machines, which could greatly degrade the performance of the algorithm (Abramson et al. 1993; Hauser and Manner 1994).

Nevertheless, master-slave parallel GAs are easy to implement and can increase the efficiency of GAs when the evaluation is computationally expensive.

83

Fine-grained parallel GAs, shown in Figure 2.12(b), also consists of one single population. Individuals from the single population are spatially structured, so that, ideally, there is only one individual for every processor. In Figure 2.12(b), each black dot represents an individual processing unit. The spatial structure limits the interactions between individuals. An individual can only compete and mate with its neighbours to produce new offspring. However, since the neighbourhoods overlap with each other, good solutions may disseminate across the entire population. Two problems need to be considered when designing a fine-grained parallel GA. First, what kind of spatial structure should be used for the population? A 2-Dimensional grid configuration, as shown in Figure 2.12(b), is very common in a fine-grained parallel GA since in most parallel computers their processing units are connected using this topology (Manderick and Spiessens 1989). However, because parallel computers also use global routers to send messages to any processor on the network, it is also possible to simulate other network topologies on the top of the grid. Anderson and Ferris (1990) experimented with the ring, hypercube, mesh and island structures and concluded that the ring and island structure were the best. Schwehm (1992) tested different population structures including ring, torus, cube and hypercube on a graph-partitioning problem, where the torus structure achieved the best performance.

Another issue to be considered is the size and the shape of the neighbourhoods.

Manderick and Spiessens (1989) observed that the performance of the algorithm degraded as the size of the neighbourhood increased. Sarma and Jong (1996) found that the ratio of the radius of the neighbourhood to the radius of the whole gird is a critical parameter in determining selection pressure for reproduction. Successful

84

applications of fine-grained parallel GAs can be found in job shop scheduling problems (Tamaki and Nishikawa 1992) and 2-dimensioanl bin-packing problems (Kroger et al. 1993).

Multiple-population parallel GAs, also called distributed genetic algorithms (DGAs) or island GAs, are the most popular forms of parallel genetic algorithms. Two important characteristics differentiate multiple-population from other types of parallel GAs: the use of several relatively isolated subpopulations and migration among subpopulations. As shown in Figure 2.12(c), each black node represents a subpopulation, or called a deme. By dividing a single population into subpopulations, the selection and mutation process of DGAs is restricted within a subset of individuals. For single population GAs, the selection and reproduction use global information over the entire population. For multiple-population parallel GAs, the selection and reproduction only uses local information within the subpopulation. This design of parallel genetic algorithms essentially mimics the spatial separation of conspecifics in nature. As in nature, the subpopulations in distributed GAs evolve separately with occasional migration between neighbouring populations. While we are trying to make genetic algorithms faster through parallelisation, we also need to make sure the performance of the genetic algorithm does not degrade due to the localisation of the search, i.e. search within subpopulations. In DGAs, the migration of good solutions among neighboured subpopulations plays a critical role. Problems such as how often the migration should happen (the migration interval), how many individuals should be exchanged between subpopulations (the migration rate), and

85

which individuals should be chosen for the migration determines the performance of distributed genetic algorithms. Pettey et al. (1987) developed a multi-population parallel GA where a copy of the best individual found in each deme is sent to all its neighbours after every generation. Their parallel GA was able to find solutions of the same quality as a sequential GA. Tanese (1989) conducted experimental studies on the frequency of migrations and the migration rate, and concluded that migration significantly increases the quality of solutions for distributed GAs. Migrations do not necessarily happen at predetermined constant frequencies. Braun (1990) described a DGA for the travelling salesman problem where migration occurred after the demes converged completely and the migration restores diversity into the demes to prevent premature convergence. Applications of multi-population parallel GAs can be found in function approximations (Muhlenbein et al. 1991), graph partitioning problems (Levine 1994), and synthesis of VLSI circuits (Davis et al. 1994) etc.