Operator optimized PSO’s

The class of operator optimized PSO’s try to improve the update equations of the PSO algorithm itself. The consideration here is that differently structured search spaces of optimization problems should get a representation in how particles surf across these structures. In a fuzzy structure with slight variations with regard to functional values (“many multimodal” problems), probably a uniform and broad flight and speed of the respective particles is more appropriate.

In contrast, in a clear defined structure optimization problem (unimodal problems) with a strong ascending or descending surface the PSO do not have to search broadly but rather very fast because of the nature of the underlying problem surface. In addition, in dynamic optimization problems there may be the need to react fast and quickly according to the speed and direction of the particles and the swarm.

The next two PSO variants will give an example how to solve these challenges with changes in the operator equation of the PSO itself. Two ways seem to be appropriate:

o Individual Enhancements to the particles update equation o Automatic Enhancement to the particles update equations

2.1.4.1 Comprehensive Learning Particle Swarm optimization (CLPSO)

The CLPSO algorithm describes itself in (Liang & et al., 2006). A major issue what CLPSO addresses is the solution finding efficiency for multimodal problems. The algorithm has the following changed velocity update equation compared to the original PSO. It is therefore an example for an operator optimized PSO.

v_id = 𝓌 v_id+ c∗φ_id( pbestfi(d)− xid) ⩝ d = {1...D}

Equation 15: CLPSO – Changed velocity update equation

with

𝑓𝑖(𝑑) = {𝑓𝑖(1), 𝑓𝑖(2), … , 𝑓𝑖(𝐷) } – defines which particles ’Pbests’ with regard to the

dimension d the particle i should follow. In this case, the flight is not the classical way, which goes iteration by iteration, but it is rather a method where the particle i parses all dimensions with the PSO algorithm. All dimensions the particle is associated with, potentially determine the overall fitness of a particle. This is a characteristic, which the algorithm can use to find excellent solutions. First, a learning probability defines whether

something is to learn or not. In addition, as long there is something to learn, the algorithm generates two functional values within the same dimension for the same particle normed to the population size ps. These functional values compare to each other and the largest value is stored in a variable. Then the particle “surfs” on to his next dimension to do the same in the next dimension and so on. All particles will search for their optimum in the same way and subsequently will find the global optimum.

The dimensional flight of the particle does show his advantages in multimodal functions and show a good diversity of the swarm. However, on the other site in high dimensional problems with simpler functions this way of updating the velocity is suboptimal. The main reason for this is that CLPSO do have a larger search range than SPSO. The more complex the problem is (higher dimensions, complex surface) the better CLPSO seems to work. The flowchart is referenced from Liang et al. (2006, p. 283) and the particles’ dimension surfing is there described as following:

Figure 5: CLPSO – dimension surfing based velocity updating of a particle

2.1.4.2 Escape Velocity Particle Swarm optimization (EVPSO)

The EVPSO algorithm is initially described by a Chinese research team from Wang et al. (2006) and it shows another example of operator optimized PSO’s. The major intend of the algorithm is to avoid trapping into local optima. It uses the regular velocity update formula Equation 1: SPSO – Particles velocity and removes the velocity update of the last

𝑣_𝑒 = {

𝑣𝑖,𝑗(𝑡) , |𝑣𝑖,𝑗(𝑡)| > 𝑒𝑐

𝑟𝑗 ⨯ 𝑣𝑖,𝑗(𝑡) | ⍴ |𝑣𝑖,𝑗(𝑡)| < 𝑒𝑐

Equation 16: EVPSO – Escape Velocity term for particle

In this context, r_j are random numbers from a uniform distribution within the interval of [-1, 1], ⍴ is a scaling factor that defines a region relevant for the escape velocity and e_c is a configured parameter which actually decides when the escape case happens (e_c < 1). In the situation, when many particles are stuck within a local optimum, the stochastic escape velocity then actually increases the likelihood to create a velocity larger than the basin diameter where the particles are potentially stuck in. Performance of EVPSO directly correlates with the parameters and e_c and ⍴. A large value of e_c shortens the time to escape which is equal to again perform a global search. A low value leads exactly to the opposite. For the escape case of the particle i ⍴ need to have a large value, which actually reduces the escape domain for the particle (particle gets faster out of the basin). Whereas this parameter is optimal for the escape case of the algorithm (large ec, large ⍴ ), during the

regular run these setting can be very suboptimal.

For the desired behavior of a balanced exploration-exploitation, the algorithm works in two phases, at the first stage, e_c is set at a large value, and ⍴ is assigned a small value (broad exploration to look for good minima), at the last stage, e_c is set at a small value, and ⍴ is set at a large value (fine granular search).

With these settings the particles make very large movements at the beginning and scan the whole solution space for candidate solutions in the early stage, and they perform a fine grain search in the final stage.

The critic to this approach is the following. Although it might be possible to escape from local minima in early stages of the algorithmic search, this capability “is bought” by the need for manual tuning of e_c, ⍴. It is very difficult to find good criteria on how to do the parameter settings automatically when the variety of benchmark problems are considered. Also in dynamic problems where the problem surface changes over time, it is hard to imagine that this approach leads to good results without tuning these parameters repeatedly.