An introduction to co-evolutionary algorithms

Co-evolution refers to a process that simultaneously evolves different species. In co- evolution, the fitness of a solution in one species is measured based on its cooperative or competitive interactions with others (Goh et al.,2010). In cooperative co-evolution, different species evolve in a mutually beneficial relationship. In competitive co-evolution,

different species have opposing interests and the success of one species depends on the failure of others.

A number of co-evolution based MOEAs have been proposed. These are briefly reviewed next. Following that, possible issues of implementation of co-evolution based algorithms are introduced.

Cooperative co-evolutionary algorithms

In cooperative co-evolution, an individual is formed by combining sub-individuals from different species (sub-populations). Sub-individuals are ranked based on the fitness of the combined individual. To better understand cooperative co-evolution based algorithms, we briefly describe some representative studies.

Potter and Jong (2000) proposed a cooperative co-evolution based genetic algorithm (CCGA). In CCGA a sub-population is initialized for each decision variable. Each individual sub-population is evolved using a traditional GA. Two versions of CCGA (i.e., CCGA1 and CCGA2) are implemented. The difference between the two versions is as follows. In CCGA1 a sub-individual is evaluated by combining it with the best known sub-individual from the other sub-populations. In CCGA2 a sub-individual is evaluated by combining it either with a random or the best sub-individual from the other sub-populations. The best fitness of the two individuals is assigned as the fitness of the current sub-individual.

However, CCGA only applies to single-objective optimisation. Keerativuttitumrong et al. (2002) proposed a multi-objective cooperative co-evolutionary genetic algorithm (MOCCGA) based on CCGA2. The only difference is that the multi-objective optimizer, MOGA, is applied to optimise each sub-population. In a similar way,Maneeratana et al. (2006) applied this cooperative co-evolution concept to NPGA and NSGA, andIorio and Li (2004) applied this concept to NSGA-II. All the extended versions are reported to exhibit significant improvements over their canonical counterparts.

Coello Coello and Sierra (2003) proposed an alternative co-evolutionary MOEA called CO-MOEA. In their approach, some promising search spaces are identified during the search, and more search effort is distributed on these regions. Specifically, the search space is divided into some pre-defined sub-regions. The same number of individuals is initialised for each sub-region. The number of individuals in each sub-region is modified in proportion to the number of new non-dominated solutions discovered in the optimisa- tion process. CO-MOEA is demonstrated to outperform PAES and NSGA-II on 2 and 3-objective problems. However, obviously the performance of CO-MOEA depends on the choice of sub-regions.

Competitive co-evolutionary algorithms

In competitive co-evolution, an inverse fitness interaction exists between different species, and different species compete to survive.

Laumanns et al. (1998) were the first to incorporate the concept of competitive co- evolution into MOEAs. Their proposed algorithm is based on the predator-prey model. Candidate solutions (acting as preys) are placed on an undirected and connected graph. Objective functions (act as predators) are selected at random to evaluate all the candi- date solutions. In the neighbourhood of an objective function, the candidate solution that has the worst objective value is replaced by its offspring.

InBarbosa and Barreto(2001), another competitive co-evolutionary algorithm was pro- posed for solving a graph layout problem. In their approach, two populations are co- evolved – a graph layout population (candidate solutions) and a set of weight vectors. The two populations are evolved by a separate genetic algorithm in a round robin process. First candidate solutions are evolved for several generations while the weight vectors are kept constant. Then a decision-maker is asked to select his/her preferred solutions. These selected candidate solutions are then held constant while the weight vectors are evolved in an attempt to produce suitable weights that can reveal the decision-maker’s preference.

Lohn et al.(2002) proposed an interesting competitive co-evolutionary algorithm called CGA, in which candidate solutions are co-evolved with target vectors. In CGA, the tar- gets gain higher fitness by being satisfied by fewer candidate solutions, and the candidate solutions gain higher fitness by meeting as many targets as possible. Experimental re- sults show that CGA performs better than SPEA and NSGA on 2-objective problems. However,Goh and Tan(2009) reported that CGA might be sensitive to the Pareto front geometries.

Combined cooperative and competitive co-evolutionary algorithms

There are also some algorithms that incorporate both competitive and cooperative co- evolutions, for example, the competitive-cooperation coevolutionary algorithm (COEA) Goh and Tan (2009). In COEA, sub-individuals are co-evolved both competitively and cooperatively. In the competitive mode, sub-individuals compete with each other to become a representative of a particular sub-population. In the cooperative mode, rep- resentatives from each sub-population co-evolve to generate better individuals. COEA is tested on three benchmark problems. Experimental results indicate that COEA per- forms well on problems with severe parameter interactions. A similar study is conducted by Goh et al. (2010) where this concept is applied to a multi-objective particle swarm optimisation (MOPSO) algorithm.

Possible issues of co-evolutionary algorithms

Co-evolution based algorithms potentially face the following issues the Red Queen Effect, cycling and disengagement (De Jong and Pollack,2004;Bongard and Lipson,2005).

(i) The Red Queen Effect means that the subjective fitness of the populations improves during the course of evolution but there is no consistent progress along the objective metric. Conversely, there is progress along an objective fitness but the improvement cannot be reflected by their subjective fitness. For example, consider students’ scores in examinations and their true ability. A series of difficult examinations may result in a low score (subjective fitness) while in fact their ability is improved. Alternatively, students may get high scores in a series of easy examinations but such scores do not represent their abilities.

(ii) Cycling refers to a scenario that solution a is better than solution b in one sub- jective metric, and simultaneously,a is worse thanb in another subjective metric. Cycling might occur when the subjective fitness criteria keeps changing over time. Cycling might arise in the situation that solutions keep searching the same space. (iii) Disengagement refers to a loss of fitness gradient in the process of co-evolution. That is, one population is completely better than another population; the sub- jective fitness of solutions becomes constant in such a way that selection becomes random. Disengagement often leads to over-specialization, that is, candidate solu- tions only progress on some objectives.

Overall, although co-evolution is an interesting and promising concept, and some useful algorithms have been developed based on this concept, a particular attention should be paid to the potential for pathologies when implementing a co-evolution based algorithm.