Chapter 2: Evolutionary Based Optimisation and Fuzzy Theories A
2.1.2 Multi-Objective Optimisation
Multi-objective optimisation [Sawaragi et al. 1985; Steuer 1986], also known as multi-criteria optimisation, is the process of simultaneously optimising two or more conflicting objectives subject to certain constraints.
In mathematical terms, the multi-objective problem can be written as follows: Find a vector x* that will optimise the following vector function:
)] ( , ), ( ), ( ), ( [ ) (x f1 x f2 x f3 x f x f k
Chapter 2: A Background
subject to the m inequality constraints gi(x) 0, i 1,2,3, ,m, and the p equality constraints hj(x) 0, j 1,2,3, ,p, where x is the vector of decision variables.
If a multi-objective problem is well formed, there should not be a single solution that simultaneously optimises each objective to its fullest. Meanwhile, multiple solutions exist, for which each objective has been optimised to the extent that if one tries to optimise it any further, then the other objective(s) will suffer as a result. These solutions are regarded as the answer to the multi-objective optimisation problem, which are called Pareto-optimal solutions or non-dominated solutions [Sawaragiet al. 1985; Steuer 1986; Deb 2001].
Pareto-optimal solutions (non-dominated solutions) are those for which improvement in one objective can only occur with the worsening of at least one other objective. They are defined as follows (for a minimisation problem):
One solution is Pareto optimal, where A is the feasible solution space, if for every
A x
A
x
either fi(x) fi(x ), i k, where k is the number of objectives, or there is at least one such that. k i ) ( ) (x f x fi i
The above definition means that x* is Pareto optimal if there is no feasible vector
x that would decrease some objective values without causing a simultaneous increase in at least one other objective value [Coello Coello 1999].
Multi-objective optimisation problems can be found in various fields such as product and process design, finance, aircraft design, the oil and gas industry, automobile design, or wherever optimal decisions need to be taken in the presence of trade-offs between two or more conflicting objectives, for instance, maximising the profit and minimising the cost of a product; maximising the performance and minimising the fuel consumption of a vehicle; and minimising the weight while maximising the strength of a particular component.
2.2 Evolutionary Computation
Evolutionary computation [Eiben & Smith 2003; De Jong 2006] is a subfield of artificial intelligence that involves optimisation problems. It uses iterative progress with a growth or development in a population. This population is then selected in a guided random search using parallel processing to achieve the desired end. Such processes are often inspired by biological mechanisms of evolution.
Evolutionary techniques mostly involve evolutionary algorithms (comprising genetic algorithms [Mitchell 1996], evolutionary programming [Eiben & Smith 2003], evolution strategy [Beyer 2001], genetic programming [Langdon & Poli 2002] and learning classifier systems [Bull & Kovacs 2005]), swarm intelligence (comprising ant colony optimisation [Dorigo & Stutzle 2004] and particle swarm optimisation [Kennedy et al. 2001]), self-organising maps [Kohonen 2001], differential evolution [Price et al. 2005], and artificial immune systems [De Castro
Chapter 2: A Background
& Timmis 2002].
2.2.1 Evolutionary Algorithms
In artificial intelligence, evolutionary algorithms (EA) are the generic population- based metaheuristic optimisation algorithms, which are a subset of evolutionary computation. An EA uses some mechanisms (operators) inspired by biological evolution: reproduction, mutation, recombination, and selection. Candidate solutions to the optimisation problem play the role of individuals in a population, and the cost function determines the environment within which the solutions live. Evolution of the population then takes place after the repeated application of the above operators.
Evolutionary algorithms consistently perform well in approximating solutions to all types of problems because they do not make any assumptions about the underlying fitness landscape. This generality is shown by successes in fields as diverse as engineering, art, biology, economics, marketing, genetics, operations research, robotics, social sciences, physics, politics, and chemistry.
2.2.1.1 Genetic Algorithms
algorithms. They are inspired by evolutionary biology such as inheritance, mutation, natural selection, and recombination (or crossover). Over the last two decades, GAs have been extensively used as search and optimisation tools in various problem domains, including science, commerce and engineering. The main reasons for their success lie in their broad applicability, ease of use and global perspective [Goldberg 1989].
The concept of a genetic algorithm was first proposed by John Holland of the University of Michigan in 1975 [Holland 1975]. Most of the initial research work can be found in various early international conference proceedings and several textbooks [Goldberg 1989; Holland 1975; Michalewicz 1992; Back et al. 1997]. Some journals are dedicated to promote research in evolutionary algorithms, certainly including GAs, such as Evolutionary Computation Journal published by MIT Press, Transactions on Evolutionary Computation published by IEEE and
Genetic Programming and Evolutionary Computation published by Kluwer Academic Publishers. New developments about GAs and other evolutionary algorithms can be found in these journals as well as in recent international conference proceedings.
2.2.1.2 Evolution Strategies
The idea behind Evolution Strategies (ES) represents a joint development of Bienert, Rechenberg and Schwefel in the 1960s at the Technical University of
Chapter 2: A Background
Berlin (TUB) in Germany. The first application of ESs was experimental and attempted to solve the shape optimisation of a bended pipe, the drag minimisation of a jointed plate and the shape optimisation of a flashing nozzle. Thereafter, different versions of ESs were suggested, such as multi-membered ESs, recombinative ESs and self-adaptive ESs. More details can be found in [Michalewicz 1992; Schwefel & Rudolph 1995; Back et al. 1997].
Though the ESs working principle is similar to that of a real-parameter GA used with selection and mutation operators only. The early ES is fundamentally different from the early binary GAs in mainly two ways: (1) ESs use real values for coding and (2) ESs do not use any crossover-like operator. In addition, the step size of ESs mutation operator can adjust itself adaptively during the optimisation process. That gives ESs the capability of self-adaptation which GAs do not have. The latter is also the reason why there are still many researchers interested in ESs.
2.2.2 Swarm Intelligence
Swarm intelligence (SI) is an artificial intelligence based on the collective behaviour of decentralised, self-organised systems. The expression was introduced by Gerardo Beni and Jing Wang in 1989, in the context of cellular robotic systems [Beni & Wang 1989].
locally with one another and with their environment. The agents follow very simple rules, and although there is no centralised control structure dictating how individual agents should behave, local interactions between such agents lead to the emergence of complex global behaviour. Natural examples of SI include ant colonies, bird flocking, animal herding, bacterial growth, and fish schooling. The most successful algorithms employing swarm intelligence are Ant Colony Optimisation (ACO) [Dorigo et al. 1996] and Particle Swarm Optimisation (PSO) [Kennedyet al. 2001].
2.2.2.1 Particle Swarm Optimisation (PSO)
Particle swarm optimisation is a population-based evolutionary computing algorithm for problem solving. It is the type of swarm intelligence that is based on social-psychological principles and provides insights into social behaviour, as well as contributing to engineering applications. The particle swarm optimisation algorithm was first described in 1995 by James Kennedy and Russell C. Eberhart [Eberhart & Kennedy 1995; Kennedy & Eberhart 1995] and the techniques have evolved greatly since then.
Since its introduction in 1995, the PSO method has become very popular due to its simplicity of implementation and ability to quickly converge to a reasonably good solution. A fair amount of research results have been reported in the literature and the first book dedicated to PSO [Kennedy et al. 2001] has been published in 2001.
Chapter 2: A Background
2.2.3 Multi-Objective Optimisation Algorithms within Evolutionary