PARTICLE SWARM OPTIMIZATION-
EVOLUTION, OVERVIEW AND
APPLICATIONS
Dr.R.UMARANI, Associate professor, Department of Computer Science,
Sri Saradha College for Women, Salem.
V.SELVI
Lecturer in Computer Science Department, Nehru Memorial College,
Puthanampatti. [email protected]
Abstract
The present work interprets on Particle Swarm Optimization and simple software agents so called particles, move in the explore breathing space of an optimization problem. The position of a particle represents a solution to the optimization problem at hand. Each particle searches for better positions in the search space by changing its velocity according to rules originally inspired by behavioral models of bird flocking. The outlines of the paper explicate the overview, evolution, applications and current trends of the PSO make use of further research.
Keywords: Particle swarm optimization, Swarm intelligence and PSO algorithms
1. Introduction
Swarm intelligence describes the collective behavior of decentralized, self organized natural or artificial systems. Swam intelligence model were employed in artificial intelligence. The expression was introduced in the year 1989 by Jing wang and Gerardo Beni in cellular robotic systems. Swarm Intelligence (SI) was a innovative pattern for solving optimizing problems. SI systems are typically made up of populations of simple agents interacting locally with one another and with their environment. The agent follows simple rules and the interactions between agents lead to the emergence of “intelligent” global behavior, unknown to the individual agents. Examples of SI include ant colonies, bird flocking, animal herding, bacterial growth and fish schooling.
The example algorithms of Swarm Intelligence are i) Ant Colony Optimization ii) Particle Swarm Optimization iii) Gravitational Search Algorithm iv) Stochastic diffusion search.
Particle Swarm Optimization belongs to the class of swarm intelligence techniques that are used to resolve the optimization problems.
2. Evolution of PSO
PSO was introduced by Kennedy and Eberhart (1995). It has roots in the simulation of social behaviors using tools and ideas taken from computer graphics and social psychology research.
In 1990, Heppner and Grenander included a roost that was attractive to the simulated birds. Both models inspired the set of rules that were used at this time on original PSO algorithm.
Social psychology research, find out the dynamic theory of social impact in the year 1990 by Nowak, Szamrej and Latane was another source of inspiration in the development of the first PSO algorithm in the year 2006 by Kennedy. The rules that govern the movement of the particles in a problem’s search space can also be seen as a model of human social behavior in which individuals adjust their beliefs and attitudes to validate with Kennedy & Eberhart peers.
3. Overview of PSO
Particle Swarm Optimization (PSO) is a population based stochastic optimization technique developed by Dr.Eberhart and Dr.Kennedy in 1995, inspired by social behavior of bird flocking or fish schooling.
The main strength of PSO is its fast convergence, which compares with many global optimization algorithms like Genetic algorithms, Simulated Annealing and other global optimization algorithms.
Particle Swarm Optimization shares many similarities with evolutionary computation techniques such as Genetic Algorithms. The system is initialized with a population of random solutions and searches for optima by updating generations. PSO has no evolution operators such as cross over and mutation. In PSO, the potential solutions called particles fly through the problem space by following the current optimum particles.
PSO is a global optimization algorithm for dealing with problems in which a best solution can be represented as a point or surface search in n-dimensional space.
Hypotheses are plotted in this space and seeded with an initial velocity as well as a communication channel between the particles. Particles then move through the solution space and are evaluated according to some fitness criterion following each time step. The particles were accelerated in the direction of communication grouping which have better fitness values. The main advantage of such approach great global minimization strategies such as simulated annealing is that the large number of members that make up the particle swarm formulate the technique impressively flexible to the problem of local minima.
Each particle keeps track of its coordinates in the problem space which are associated with the best solution it has achieved so far. This value is called pbest. Another best value that is tracked by the particle swarm optimizer is the best value, obtained so far by any particle in the neighbors of the particle. This location is called lbest. When a particle takes all the population as its topological neighbours, the best value is a global best and is called gbest.
The particle swarm optimization concept consists of each time step, changing the velocity of each particle toward its pbest and lbest locations. Acceleration is weighted by a random term, with separate random numbers being generated for acceleration toward pbest and lbest locations.
4. Parameters of PSO
Selecting good PSO parameters has been the subject of research. A simple way of tuning the PSO parameters was presented by Pedersen et al (2000) who also made numerous experiments with different optimization problems and settings. The technique for tuning PSO parameters is called meta-optimization because another optimization method is used in an overlaying manner to tune the PSO parameters.
The inertia weight is employed to control the impact of the previous history of velocities on the current one. During the iteration time t, the update of the velocity from the previous velocity to the new velocity is determined by the following equation(1)
Vij(t+1)= w Vij(t)+c1r1(Xij (t) – Xij(t)) + C2r2(Xj *(t)-Xij(t)) (1)
The new position is then determined by the sum of the previous position and the new velocity by the following equation(2)
Xij(t+1) = Xij(t)+Vij(t+1) (2)
experience which is the memory of the best past position and the experience of its most successful particle in the swarm.
The parameter w regulates the trade-off between global (wide ranging) and local (near by) exploration abilities of the swarm. A large inertia weight facilitates global exploration while a small one tends to facilitates global local exploration. A suitable value for the inertia weight w usually provides balance between global and local exploration abilities and consequently results in a reduction of the number of iterations required to locate the optimum solution. Initially, the inertia weight is set as a constant. Thus, an initial value is around 1.2 and gradually reducing towards 0 can be considered as a good value for w. The parameters C1=C2=2 can be set as default values. Some experiment result indicate that C1=C2=1.49 might be provide even better results swarm size value might be 20. 5. Variants
The Particle Swarm Optimization algorithm has undergone a number of changes since it was first proposed. Most of these changes affect the way the velocity of a particle is updated. New PSO variants are continually being introduced in an attempt to improve optimization performance. There are certain trends in that research, one is to make a hybrid optimization method using PSO combined with other optimizers, trend is to try and alleviate optimization stagnation by some mechanism. For example, reversing the movement of the PSO particles.
5.1 Discrete PSO
Most particle swarm optimization algorithms are designed to search in continuous domains. There are a number of variants that operate in discrete spaces. The first variant proposed for discrete domains was the binary particle swarm optimization algorithm. In this algorithm, a particle’s position is discrete but its velocity is continuous.
5.2 Constriction Coefficient
The Constriction Coefficient was introduced as an outcome of a theoretical analysis of swarm dynamics.
5.3 Bare-bones PSO
The Bare-bones particle swarm is a version of the particle swarm optimization algorithm in which the velocity and position update rules are substituted by a procedure that samples a parametric probability density function.
5.4 Fully informed PSO
In the standard particle swarm optimization algorithm, a particle is attracted toward its best neighbor. A variant in which a particle uses the information provided by all its neighbors in order to update its velocity is called the fully informed particle swarm.
6. Applications and Current Trends
The first practical application of PSO was in the field of neural network training and was reported together with the algorithm itself. Many applications have been explored ever since, including telecommunications, control, data mining, design, combinatorial optimization, power systems, signal processing and many others. Till date, there are hundreds of publications reporting applications of particle swarm optimization algorithms.
Swarm intelligence-based techniques can be used in many applications. The U.S. Military is investigating swarm techniques for controlling unmanned vehicles.
The European Space Agency is thinking about an orbital swarm for self assembly and interferometer. NASA is investigating the use of swarm technology for planetary mapping.
6.1 Artificial Neural Network and PSO
An artificial neural network is an analysis paradigm that is a simple model of the brain and the back-propagation algorithm is the one of the most popular method to train the artificial neural network. Recently, the significant research efforts were to apply Evolutionary Computation (EC) techniques for the purposes of evolving one or more aspects of artificial neural network.
Evolutionary computation methodologies have been applied to 3 main attributes of neural networks. Network connection weight, network architecture and network learning algorithms most of the work involving the evolution of Artificial Neural Network has focused on the network weight and topological structure was encoded as chromosome in GA. The selection of fitness function depends on the research goals. For a classification problem, the rate of misclassified patterns can be viewed as the fitness value ( Ref 7).
The advantage of the EC is that EC can be used in cases with non differentiable PE transfer functions and no gradient information available. The disadvantages are:
1. The performance is not competitive in some problems.
2. Representation of the weights is difficult and the genetic operators have to be carefully selected or deselected.
The following is the simple example of evolving ANN with PSO. The problem is a benchmark function of classification problem. Measurement of four attributes of iris flowers are provided in each data set record; sepal length, sepal width, petal length and petal width. Fifty set of measurements are present for each of three varieties of iris flowers, for a total of 150 records.
A 3-layer ANN is used to do the classification. There are 4 inputs and 3 outputs. So the input layer has 4 neurons and the output layer has 3 neurons. One can build up the number of hidden neurons. Suppose the hidden layer has 6 neurons, we can evolve other parameters in the feed-forward network to evolve the weight. So the particle will be a group of weights, there are 4*6+6*3= 42 weights. So the particle consists of 42 real numbers. The range of weight can be set to [-100,100]. After encoding the particles, we need to determine the fitness function. To classify the problem, we feed all the patterns to the network whose weight is determined by the particle, get the outputs and compare it with the standard output. Then we record the number of misclassified pattern as the fitness value of the particle. Indeed, we can apply to train the ANN to get lower number of misclassified patterns as possible. Further, ANN need to adjust the lower number of hidden layers and the range of the weights to get better results in different trails.
6.2 PSO for Data Mining
Data Mining and PSO do not have many properties in universal. However they can be used together to form a method which often leads to formulate the result, even when other methods would be too expensive or difficult to implement. Ujjinn and Bentley (2006) provided internet-based recommender system, which employs a particle swarm optimization algorithm to learn personal preferences of users and provide tailored suggestions. The basic idea of combining PSO with data mining is simple. To extract this knowledge, a database may be considered as a large search space and a mining algorithm as a search strategy. PSO makes use of particles moving in an n-dimensional space to search for solutions for an n-variable function optimization problem. The datasets are the sample space to search and each attribute is a dimension for the PSO-miner.
During the search space procedure, each particle is evaluated using the fitness function. The fitness function measures the predictive accuracy of the rule for data mining and it is given by the following equation: Predictive accuracy = (A ^ C| - 1/2) )/ |A|
Where, | A ^ C| is the number of examples that satisfy both the rule antecedent and the consequent and |A| is the number of cases that satisfy only the rule antecedent. PSO usually search the minimum for the problem space considered.
Rule pruning is a common technique in data mining. The main goal of rule pruning is to remove irrelevant terms that might have been unduly included in the rules.
The performance of PSO-miner was evaluated using four public-domain data sets. The used parameter settings are as follows:
Swarm size = 30 C1=C2=2
Maximum uncovered cases = 10 Maximum number of iterations=4000 6.3 Job Scheduling on Computational Grids
Grid Computing is a computing framework to meet the growing computational demands. To illustrate the performance of the algorithm, we consider a finite number of grid nodes and assumed that the processing speed of the grid nodes and the job lengths are known. The specific parameters for PSO are swarm size, self-recognition co-efficient, social coefficient & inertia weight. The algorithm was repeated 10 times. With different random seeds each trial had a fixed number of 50*m*n iterations (Ref 1). The makespan values of the best solutions throughout the optimization run were recorded. The averages and the standard deviations were calculated from the co different trials. The standard deviation indicates the differences in the results during the 10 different trials.
We tested a job scheduling problem involving 3 machines and 13 jobs and 5 machines and 100 jobs. The results of 10 PSO runs were:
{46,46,46,46,46.5,46.5,46,46.5,46.6667} with an average value of 46.2667. PSO provided the best results 5 times. 7. Conclusion
To conclude, Particle swam optimization is a new heuristic optimization method based on swarm intelligence. Compared with the other algorithms, the method is very simple, easily completed and it needs fewer parameters, which made it fully developed. To overview, evolution, parameter and the applications of PSO are presented in a simple way. Although PSO has been used mainly to solve unconstrained, single objective optimization problems, PSO algorithms have been developed mainly to solve constrained problems, multi objective optimization problems and problems with dynamically changing landscapes and to find multiple solutions.
References
[1] A.Abraham et al: Swarm intelligence: Foundations, perspectives and applications. Studies in Computational Intelligence(SCT)26,3-25(2006)
[2] Kennedy,j. Eberhart R.C.(2001). Swarm intelligence, Morgan Kaufmann. ISBN 1-5586-595-9.
[3] Bratton,d.; Blackwell,T.(2008) “A Simplifies recombinant PSO” journal of Artificial Evolution and applications.
[4] Shi, Y.Eberhart, R.C. (1998) “Parameter Selection in Particle Swarm Optimization”, proceedings of evolutionary programming VII. [5] Eberhart, R.C, Shi, Y (2000). Comparing Inertia weights and constriction factors in Particle Swarm Optimization. Proceedings of the
Congress on evolutionary competition. Pp- 84-88.
[6] Trelea, I.C. (2003). The Particle Swarm Optimization Algorithm: convergence analysis and parameter selection. Information processing letters 85:317-325
