Evolutionary Algorithms (EAs) are a set of algorithms that model problems as an evolu- tionary process and implement the characteristic events such as mutation, recombination,
and population seeding. Evolutionary Algorithms (EAs) are ideal for exploring complex search spaces.
In this section, we highlight only some evolutionary algorithms briefly, as it is not the primary focus of the research. Firstly, the Genetic Algorithm (GA) which is one of the most widely used in optimization problems, with so many variations that extend from it [39, 74]. Afterward, the working principles of another evolutionary algorithm, Differential Evolution (DE) is explained.
2.3.1
Genetic Algorithms (GA)
The Genetic Algorithm (GA) is a widely popular population-based algorithm that is bio- logically plausible. The concept of this algorithm is basically to form a representation of the problems parameters that are then encoded onto an array, commonly referred to as the chromosome, and adopt evolutionary operators such as mutation and cross-over to find an optimal solution. There are various representation schemes possible, though these can all be classified as binary or real value representations [107, 109]. In binary representational schemes, these chromosomes will represent the parameters of the problem in binary, while in real values representations, real values are used to represent parameters. Depending on the number of parameters and encoding approach, the chromosomes will vary in their length. However, canonical genetic algorithms use binary representations of a problem, so the parameters have to adopt an approach that encodes parameters onto a chromosome as binary representations. Such representations typically have to compensate with relatively complicated encoding methods so that the length of the chromosome is not too long -as this complicates the search space, and not too short that it causes a loss of weight preci- sion, which will significantly affect the accuracy of approximating the mapping function (i.e underlying function of the problem) [112, 109, 108, 111].
However, in the case of real value representations [98, 4, 11], the precision of weights can be modified without altering the length of the chromosome. Standard evolutionary operators have to be modified to deal with these changes.
Once the problem has been represented in the form of a chromosome, a finite popu- lation of m chromosomes with different parameter variations is generated randomly and encoded, P = {c1, c2, c3. . . cm}. However, to ensure a better performance advantage, prior
knowledge of the context of an application can be used to optimize properties of the ini- tialization process such as the distribution of the random number generated, and range. This significantly reduces the search space complexity, thereby increasing the convergence speed. The next step in the process is to evaluate the solutions using an evaluation function, f(.). This function is another crucial element of the evolutionary process which can affect
its performance in a number of ways. One of such ways includes its ability to find optimal solutions. There is no general evaluation function for all problems, and one would ideally want to carefully craft a problem specific evaluation function, which takes into account the nature of the domain [107, 108, 112, 109]. All chromosomes or candidate solutions are scored using the evaluation function. This scoring process is the biological equivalent of the process that determines the fitness of individuals in the population. The next process in line is the selection process. It relates to the survival of the fittest trait found in biology and has two steps; the first step, selection, selects candidates that will make up the next generation of candidates solutions based on fitness. The other step, reproduction, creates new individuals that will replace weaker candidates using evolutionary operators (i.e. mu- tationand cross-over). The mutation operation alters fitter individuals selected typically at random to alter their chromosomes while the cross-over operation forms new candidate so- lutions by exchanging chunks of genes from the gene strings (chromosomes) of two parent individuals. The choice of which operator to apply can be selected on a probabilistic basis. These processes are repeatedly executed until a stopping criterion is met, which could be after a maximum number of generations (i.e. iterations), or/and if a solution in the population has reached the desired fitness level.
2.3.2
Differential Evolution (DE)
Differential Evolution is also a population-based evolutionary algorithm developed by Storn and Price (1996) as highlighted in [40]. Although, its efficiency at converging on near opti- mal solutions has yet to be formally proven as explained by Jones [58]; it has been found to be efficient and effective for a wide variety of problems. It can be considered an extension of the Genetic Algorithm. The only significant difference between differential evolution and other classical evolutionary algorithms is its approach towards mutation and recom- bination of individuals in the population. It also introduces two new parameters: (1) a weighting factor and, (2) a probability of cross-over.
In differential evolution, the mutation operation starts off by randomly selecting three solution vectors from the population. Assuming these solution vectors are labeled as Vx,Vy
and Vz; then the next step is to find the difference between the first two vectors, i.e. Vx
and Vy, which is multiplied by the weighting factor, F, and then added to the solution
vector of the third solution, i.e. Vz, resulting in the mutant solution’s vector, Vn (refer to
equation 2.6). This not only helps explore more regions of the search space as a result of its stochastic property but also results in a property found in particle swarm optimisation - another population-based algorithm that accounts for the effects of other members of the population on each individual vector - known as the flocking behavior.
Vn= Vz+ F (Vx−Vy) (2.6)
Extensions of this algorithm such the one used to evolve Neural Diversity Machines [71] implements a recombination stage which follows afterward, where the mutant solution and the third solution, Vz, create a new vector (or solution) by competing on a probabilistic
basis (i.e. probability of cross-over) to contribute to the new solution Vn. In other words,
if a random number generated is less than the probability of cross-over; a gene from the mutant vector is copied. However, If it is greater, then the gene from the third solution is copied instead. This vector is then evaluated and is only copied into the next generation if it has better fitness than the third solution. This behavior helps to relief pressure that might otherwise lead to premature convergence.