• No results found

An evolutionary game based particle swarm optimization algorithm

N/A
N/A
Protected

Academic year: 2021

Share "An evolutionary game based particle swarm optimization algorithm"

Copied!
6
0
0

Loading.... (view fulltext now)

Full text

(1)

www.elsevier.com/locate/cam

An evolutionary game based particle swarm

optimization algorithm

Wei-Bing Liu

, Xian-Jia Wang

Systems Engineering Institute, Wuhan University, Wuhan 430072, China

Received 16 September 2006; received in revised form 30 January 2007

Abstract

Particle swarm optimization (PSO) is an evolutionary algorithm used extensively. This paper presented a new particle swarm optimizer based on evolutionary game (EGPSO). We map particles’ finding optimal solution in PSO algorithm to players’ pursuing maximum utility by choosing strategies in evolutionary games, using replicator dynamics to model the behavior of particles. And in order to overcome premature convergence a multi-start technique was introduced. Experimental results show that EGPSO can overcome premature convergence and has great performance of convergence property over traditional PSO.

© 2007 Elsevier B.V. All rights reserved.

MSC: 74P99; 91A22

Keywords: Particle swarm optimization; Game theory; Evolutionary game; Replicator dynamics; Premature convergence

1. Introduction

The particle swarm optimization (PSO), first introduced in[3], is a swarm intelligence algorithm that can be likened to the behavior of a flock of birds or the sociological behavior of a group of people. As a key optimization tech-nique, PSO has been used extensively in many fields, including function optimization, neutral network, fuzzy system control, etc.

Since the introduction of the PSO algorithm, during the last decade many researchers presented a lot of improved PSOs. Shi et al. gave the PSO with inertia weight[9]. This method has great advantage of convergence property over traditional PSO. Clerc presented a new PSO with constriction factors which may help to ensure convergence [1]. Parsopoulos et al. used function “stretching” to improving PSO[6].

Game theory is a mathematical theory of socio-economic phenomena exhibiting interaction among decision-makers, whose actions affect each other. In 1970s evolutionary game theory was introduced by Maynard Smith. In his book

Evolution and Theory of Games[5] he presented an evolutionary approach in classical game theory. Evolutionary game theory has extensive applications in many fields. Ficici and Pollack introduced evolutionary game theory in the simple co-evolutionary algorithm[2]. And Wiegand et al. used evolutionary game theoretic model to help analyze the dynamical behaviors of co-evolutionary algorithms[7].

Foundation item: supported by the National Natural Science Foundation of China (60574071).Corresponding author.

E-mail address:liuweibing2002@sohu.com(W.-B. Liu).

0377-0427/$ - see front matter © 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.cam.2007.01.028

(2)

In this paper, we introduced evolutionary game theory in the traditional PSO and presented a evolutionary game based particle swarm optimization algorithm (EGPSO). We used replicator dynamics to analyze the learning and actions of birds, and in order to overcome premature convergence a multi-start technique[10]was used. Simulation results proved the algorithm’s efficiency.

The rest of this paper is organized as follows. Firstly we introduce the theory of traditional PSO in Section 2. In Section 3, we give the fundamental notion of game theory and replicator dynamics model in evolutionary games. Section 4 presents the new algorithm (EGPSO). We analyze the simulation results in Section 5. The paper ends with conclusions in Section 6.

2. Particle swarm optimization

The PSO’s operation is as follows. Each particle represents a possible solution to the optimization task. During each iteration, each particle accelerates in the direction of its own personal best solution found so far, as well as in the direction of the global best position discovered so far by any of the particles in the swarm.

Let Present denote the current position in the search space, V is the current velocity. During each iteration, each particle in the swarm is updated using the following equations:

V (t + 1) =  ∗ V (t) + c1· rand · (pBest(t) − Present(t))

+ c2· rand · (gBest(t) − Present(t)), (1)

Present(t+ 1) = Present(t) + V (t + 1), (2)

where rand ∼ U(0, 1) is the element from uniform random sequence in the range (0, 1), and c1 and c2 are the acceleration coefficients, usually c1= c2= 2.  is the weight coefficient and 0.10.9. pBest, the personal best position of each particle, is updated by

pBest(t + 1) = 

pBest if f (Present(t+ 1))f (pBest(t)),

Present(t+ 1) if f (Present(t + 1)) > f (pBest(t)) (3) and gBest, the global best position, is the best position among all particles in the swarm during all previous steps. It means

gBest(t + 1) = arg max

i f (pBesti(t + 1)) for any particle i. (4)

The value of V can be clamped to the range[−Vmax, Vmin] to ensure particles in the search space.

In this paper, the variable is the inertia weight, this means that the value of  is typically setup to vary linearly from maximum to minimum during the course of iterations, and is formulated as follows:

 = max− iter ·

max− min

itermax , (5)

where itermaxis the time of maximum iteration, and iter is the time of current iteration.

In the course of searching by using PSO, if a particle discovers a current optimal position (not global optimal point), all the other particles will move closer to it, then particles are in the dilemma of local optimal point. This is so-called premature convergence.

3. Game theory and replicator dynamics

We restrict our view to the class of finite games in strategic form. Generally a normal form game consists of three key components:

• Players: Let I = 1, 2, . . . , n be a set of players, where n is a positive integer.

• Strategies space: For each player i ∈ I , let Si denote a set of allowable actions, called the pure strategies set. The choice of a specific action si ∈ Si of a player i is called a pure strategy. The vector s= (s1, s2, . . . , sn) is called a pure strategies profile. In this paper, we only consider the pure strategies.

(3)

Replicator dynamics and evolutionary stable strategies (ESS) are the key notions in evolutionary games. Suppose strategies space S and the population share of individuals taking the jth strategy denoted by xj, wherej∈Sxj = 1.

The replicator dynamics equation is

˙xj= (fj− f ) · xj, (6)

where fj is the payoff of jth strategy and f is the average of payoffs over all individuals. This equation implies that the share of the jth strategy grows or shrinks in proportion to the difference between its payoff and the average payoff. 4. EGPSO

4.1. The principle of EGPSO

In this paper, we make maps as follows.

particles in a swarm −→ players in a game, research space −→ strategies space, fitness function −→ payoff function,

gBest −→ evolutionary stable strategy.

Therefore, all particles research the best position can be looked as players’ pursuing the maximum payoff. Consider the bounded rationality of the particles, we use replicator dynamics to model the researching process of the particles.

From Eq. (6), we can conclude:

Theorem. If we make a transformation on as follows:

f= f +  (,  > 0) (7)

thenfwill change the evolutionary velocity, but the evolutionary path is invariable. Proof. From Eqs. (5) and (6), we have

˙x = x(f− f) = x[(x + ) − x + ] = x(f − f )

consider Eq. (5), the result is obvious. 

4.2. Use multi-start to overcome premature convergence

Premature convergence is a problem that always appears when using PSO, in order to overcome the problem we use multi-start. Multi-start is the simplest technique to compute more than one maximum of a function. In this paper, we use this technique to overcome premature convergence.

Multi-start, as its name, as soon as a maximum is detected the algorithm is reinitialized in the search space. However, this approach does not guarantee that the algorithm will not convergence to one of the previously detected maximums. In order to obtain multiple maximums, we use the deflection technique[4,8]. The approach is as follows.

Let f (x) : S → R, S ⊂ Rn be the original objective function and xi, i= 1, 2, . . . , m be m maximums of f (x). Then, a new function F (x) is defined as follows:

F (x) = T1(x; x1, 1)−1· · · Tm(x; xm, m)−1f (x), (8) wherei, i=1, . . . , m, are relaxation parameters, and T1, . . . , Tmare appropriate functions in the sense that the resulting function F (x) has exactly the same maximums as f (x), except at points x1, . . . , xm. The function Ti is defined as follows:

Ti(x; xi, i) = tanh(ix − xi), i = 1, . . . , m. (9)

Therefore, when the optimization algorithm detects a maximum xi of the objective function, the algorithm is restarted and an additional Ti(x; xi, i) is included in the objective function F (x).

(4)

20 18 16 14 12 10 8 6 4 2 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 x ∝

Fig. 1. The curve of six-point function.

4.3. Proposed algorithm

The EGPSO algorithm can be summarized in the following steps:

• Step 1: Randomly choose a swarm of particles, for each particle initialize position and velocity. • Step 2: Make a transformation on f (x) using (7).

• Step 3: Let pBest be current best position, let also gBest be current best position among all particles. • Step 4: Judge whether the convergence condition is satisfied, if yes, go to step 7. Otherwise, go to step 5. • Step 5: For any particles in the swarm:

(a) Update position and velocity using (1) and (2). (b) Update pBest using (3).

(c) Update gBest using (4). • Step 6: Update the offspring by (6).

• Step 7: If the convergence condition is satisfied, go to step 7. Otherwise, turn to step 5. • Step 8: Output gBest, let gBest = x

i, define F (x) by (8) and (9), then go to step 1.

• Step 9: Output x

i, the global maximum is obtained by arg maxiF (xi).

5. Simulation experiments

In order to show the efficiency of the EGPSO, we give three experiments as the following: Six-point: f1(x) = 10 + sin(1/x) (x − 0.16)2+ 0.1, x ∈ (0, 1). (10) Rosenbrock: f2(x) = n−1  i=1 100(xi+1− xi2)2+ (1 − xi)2. (11) Sphere: f3(x) = n  i=1 x2 i. (12)

Fig. 1is the image of the function f1(x). The function is called six-point function because it has six local maximum

points, the optimum is f (x)=19.89489788973. Because of many local maximum points the phenomenon of premature convergence easily appears when using traditional PSO. Now we use EGPSO to search the maximum resolution of the function. We give the evolution curve using EGPSO and PSO inFig. 2. In the algorithm, the number of particles

N is 20, c1= c2= 2, rand = 0.8, max= 0.9, min= 0.1,  = 5,  = 0,  = 0.1. The optimal solution obtained

is f (x)max= 19.8949, and x = 0.1275, and the iteration times are no more than 20. FromFig. 2we know EGPSO

(5)

20 19 18 17 16 15 14 13 12 11 0 10 20 30 40 50 60 Iteration f(X) 100 Iteration 200 EGPSO 90% 100% PSO 83% 75% EGPSO PSO

The rate of successfully search

Fig. 2. The evolution curve of six-point function.

3000 2500 2000 1500 1000 500 0 3 2 1 0 -1 -2 -1 0 1 2 x1

Fig. 3. The curve of two-dimensional Rosenbrock function n= 2.

of successfully search we can conclude that multi-star can make particles avoid the local optimal situations, therefore EGPSO can overcome premature convergence.

Rosenbrock function f2(x) is n-dimensional.Fig. 3is the image of two-dimensional Rosenbrock function (n= 2). As is well known, when (x1, x2) = (0, 1), the function has the only minimum value f (x) = 0. Rosenbrock function is

an example that is difficult to optimize for many algorithms. Use EGPSO, N= 20, c1= c2= 2, rand = 0.8, max= 0.9,

min= 0.1,  = 10,  = 0,  = 10, less than 70 iterations we get the minimum solution f (x)min= 0 and x1= 1.0002,

x2= 1.0004.Fig. 4is the evolution curve of two-dimensional Rosenbrock function using PSO, EGPSO and GA,

respectively. Among the three algorithms, the convergence speed of EGPSO is quickest. It indicates that EGPSO has great performance of convergence property over traditional PSO and GA algorithms. In addition, experiments are conducted to compare the GA, PSO and EGPSO algorithms (in Table.1). For all the functions tested, the running time of EGPSO is shortest and the mean results obtained are best. That is to say, EGPSO yield better results than GA and PSO.

6. Conclusions

How to improve the performance of PSO is an important problem. In this paper, we connect game theory with the traditional PSO, use a multi-start technique to overcome the premature convergence, and present a new PSO

(6)

0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 f(x) 0 10 20 30 40 50 60 70 iteration PSO EGPSO GA

Fig. 4. The evolution curve of two-dimensional Rosenbrock function.

Table 1

Results by using different algorithms (GA, PSO, EGPSO)

Test function GA PSO EGPSO

Six-point Optimum 19.89489788973 Mean result 17.796 18.624 19.721 CPU time (s) 14.1 6.4 5.8 Rosenbrock Optimum 0 n = 2 Mean result 1.253 0.016 0.002 CPU time (s) 13.5 8.6 7.1 Rosenbrock Optimum 0

n = 30 Mean result 4.136e+ 6 46.734 35.663

CPU time (s) 42.5 30.2 25.6

Sphere Optimum 0

n = 30 Mean result 10.352 9.845 0.106

CPU time (s) 20.3 16.5 12.1

CPU time is the amount of running time (in seconds) used by MATLAB from the time the algorithm starts.

algorithm (EGPSO). Experiments results show that EGPSO can overcome the premature convergence and has a marked improvement in performance over the traditional PSO.

References

[1]M. Clerc, The swarm and the queen: towards a deterministic and adaptive particle swarm optimization, in: Proceedings of the Congress on Evolutionary Computation, Piscataway, IEEE Service Center, NJ, 1999, pp. 1951–1957.

[2]S.G. Ficici, J.B. Pollack, A game-theoretic approach to the simple coevolutionary algorithm,http://portal.acm.org/citation.cfm?id= 669093 . [3]J. Kennedy, R.C. Eberhart, Particle swarm optimization, in: Proceedings of the IEEE International Conference on Neural Networks, vol. 4,

IEEE Press, USA, 1995, pp. 1942–1948.

[4]G.D. Magoulas, M.N. Vrahatis, G.S. Androulakis, On the alleviation of local minima in backpropagation, Nonlinear Anal. Theory Methods Appl. 30 (1997) 4545–4550.

[5]J. Maynard Smith, Evolution and the Theory of Games, Cambridge University Press, New York, 1982.

[6]K.E. Parsopoulos, V.P. Plagianakos, G.D. Magoulas, et al., Improving particle swarm optimizer by function “Stretching”, in: N. Hasjisavvas, P. Pardslos (Eds.), Advances in Convex Analysis and Global Optimization, Kluwer Academic Publishers, The Netherlands, 2001, pp. 445–457. [7]R.P. Wiegand, W.C. Liles, K.A. De Jong, Analyzing cooperative coevolution with evolutionary game theory, Evol. Comput. 2 (2002) 1600–1605. [8]N.G. Pavlidis, K.E. Parsopoulos, M.N. Vrahatis, Computing Nash equilibria through computational intelligence methods, J. Comput. Appl.

Math. 175 (2005) 113–136.

[9]Y. Shi, R.C. Eberhart, A modified particle swarm optimizer, in: IEEE International Conference of Evolutionary Computation, Anchorage, AK, 1998.

References

Related documents

• A list of the current DDC points (if applicable); and • The current control sequences (sometimes these are on the drawings).. The first deliverable of the retrofit should be

SUMMARY : Ultraviolet irradiation of spores of three leaf-infecting fungi, Botrytis fabae, Uromyces fabae (causes of ‘chocolate spot’ and rust of broad beans, respec-

Transformation plans for children and young people’s mental health provision 1–3 focus on early identification, increasing access to services and reducing waiting times for

Nonprofit governance focuses on issues such as setting organizational direction; determining mission, vision, and values; engaging in strategic thinking and planning;

In addition to improving our understanding of how sites should be combined to estimate regional trends, refinement of catchment areas estimated using stable hydrogen isotopes

While southern and western moisture fluxes are typical of the increases of discharge of the Sava River, summer events in the Mura and Drava catchments were frequently associated

Cu(II), Zn(II) exhibit tetragonal structure were as Mo(II) and Fe(II) complexes exhibit octahedral structure.. The proposed structure of Cu (II), Zn (II), Mo (II) and Fe (II)

In this narrative review, we highlighted that there is some evidence that exogenous melatonin may alter pubertal timing in some animals, that photoperiodic and melatonin