Adaptive Novel Global Harmony Search Algorithm for Function Optimization Problems

2018 International Conference on Physics, Computing and Mathematical Modeling (PCMM 2018) ISBN: 978-1-60595-549-0

Adaptive Novel Global Harmony Search Algorithm for Function

Optimization Problems

Jia-xin CHENG

1

,

Xiang-qian LIU

1,*

and Xiao-yong CHEN

2

1_{School of Computer and information Technology, Beijing Jiaotong University, Beijing, China}

2_{General engineer room of China Railway Jinan Bureau Group Co., Ltd}

*Corresponding author

Keywords: Harmony search, Meta-heuristic, Function optimization.

Abstract. Harmony Search (HS) algorithm is a new meta-heuristic global search algorithm. To solve the algorithm easily fall into the local optimal problem, this paper proposes an adaptive novel global harmony search (ANGHS) algorithm based on NGHS algorithm. First, a new location update strategy is given using the adjustment parameter F. Then, the mutation operation is introduced in the case of excluding the small probability Pm. In order to avoid blind adjustment of the step size in the

late search and effectively adjust the structure of the solution, the definition of the norm is introduced to adjust parameter bw dynamically. In this paper, 7 standard Benchmark functions are used to simulate the experiment. The results show that the algorithm avoids the premature problem of the algorithm and enhances the global search ability of the algorithm, with obvious improvement in convergence speed and stability, and has good optimize performance.

Introduction

At present, many intelligent optimization algorithms have been applied to engineering function optimization problems [1]. Among them, several commonly used biomimetic optimization algorithms are GA, SA, ABC, ACO, PSO and DE [2]. Harmony search algorithm (HS) is another novel meta-heuristic intelligent optimization algorithm that Geem et al. proposed in 2001 by imitating the harmony process of musicians creating music [3]. The algorithm has strong searching ability, simple structure, convergence has nothing to do with the initial value of the problem variables, and is easy to combine with other algorithms. However, the harmony search algorithm in the early convergence rate is faster, the late convergence rate gradually slows down [4]. For multidimensional and complex problems, when multiple iterations are solved, precocity occurs, and local optimum is reached, and it is difficult to get rid of local optimal problems. In response to these shortcomings, scholars have improved the classical harmony search algorithm, and many variants of harmony search algorithms have emerged: HIS (Improved harmony search algorithm)[5], DSHS (Dynamic self-adaptive harmony search algorithm), GHS (Global-best harmony search algorithm)[6], IGHS[7,8], SGHS[9,10], NGHS (Novel global-best harmony search algorithm) [11,12], Improved novel global-best harmony search algorithm (INGHS) [13]. These variants of the HS algorithm have a certain improvement in terms of convergence and search capabilities, but there are still some deficiencies in the local optimal problems. Therefore, in order to avoid the algorithm falling into the local optimum, and improve the convergence speed of the algorithm, this paper proposes an Adaptive novel global harmony search algorithm (ANGHS).

The Harmony Search Algorithm

Basic Harmony Search Algorithm(HS)

The basic principle of the harmony search algorithm is to imitate the process by which musicians create music and harmony. In music creation, musicians constantly adjust the pitch of the instrument to produce best harmony effect. The specific algorithm steps are as follows:

size (HMS), harmony memory considering rate (HMCR), pitch adjusting rate (PAR), bandwidth (bw), the number of decision variables N, the number of iterations (NI).

Step 2: Initialize the harmony memory: Generate HMS optimization problem solutions

1 2

, ,... HMS

x x x _{randomly and store them into the harmony memory (HM) as the initial solution. To}

make the initial harmony population have a certain degree of uniformity and dispersion, each pitch in the harmony memory is generated by formula (1):

i i

(UB LB ) * rand(0,1)

j i i

x LB  

(1)

where, i1,2,...,N; j1,2,...,HMS, rand is a random number between (0,1), LBi _and i

UB _{are the upper and lower bounds of the decision variables i, respectively.}

Step 3: Improvise a new harmony: Generating a new harmony is called improvisation process.

HS algorithm generates a new harmony

' ' ' '

1 2

(x , x ,..., x )n

X 

. Each pitch in the new harmony

'

i x

generated using the following rules: memory consideration; pitch adjustment; random selection. Memory consideration and the new harmony are generated from the harmony memory with the probability of HMCR, random selection, the new harmony are randomly generated from the definition domain by the probability of 1-HMCR. The formula is as shown in formula (2):

   

 

 



N 1,2,..., i otherwize,

,

HMCR rand ), ,..., , ( 1 2

i j i

HMS i i i j i j i

X x

if x x x x x

(2) Adjust the pitch for the harmony from HM in the adjustment probability of PAR. The formula is as shown in formula (3):

, otherwize; i=1,2,...,N

rand * , if rand<PAR

j i j

i j

i

x BW

x x

 

   

 ₍₃₎

Step 4: Update harmony memory: if f(xnew) f(xi)_,

x

i



x

new_{; otherwise, the harmony memory} unchanged.

Step 5: Check the stopping criterion: Repeat steps 3 and 4 until the maximum number of iterations is reached, return best harmony.

The Improved Harmony Search Algorithm(IHS)

To address the shortcomings of the HS that use fixed parameters PAR and BW. The IHS algorithm was developed by Mahdavi et al. [14] The IHS algorithm and the HS algorithm use the same memory considerations, pitch adjustment and random selection methods. Only in step 3, the parameters PAR and BW are dynamically adjusted as in equation (4), (5):

max min

min

(t) PAR PAR PAR

PAR t

NI 

  

(4)

min

max max

ln( ) bw(t) exp( )

bw bw

bw t

NI

  

(5)

Dynamic Self-adaptive Harmony Search Algorithm (DSHS)

Novel Global-best Harmony Search Algorithm (NGHS)

The NGHS algorithm is proposed based on the idea of population intelligence in particle swarm optimization algorithm. The NGHS algorithm excludes three parameters of HMCR, PAR, and bw in the HS algorithm, introducing position update and genetic mutation. The NGHS algorithm position update operation is as formula (7), (8):

2 best worst

R i i

x  x x

(7)

'

() ( )

worst worst

i i R i

x x rand  x x

(8) The NGHS algorithm's position update operation is apt to fall into a local optimum, the mutation operation is introduced when the mutation probability is less than Pm. Specific operations such as formula (9):

'

() ( )

L U L

i i i i

x x rand  x x

(9) where, L i x and U i x

are the lower and upper bounds of the i-th harmony component, respectively.

Adaptive Novel Global Harmony Search Algorithm(ANGHS)

In this paper, an adaptive novel global harmony search (ANGHS) algorithm is proposed, by adjusting the parameter F to give a new location update strategy, then, excluding the introduction of mutation operations with a small probability Pm. The definition of the norm was introduced to represent the diversity of the harmony memory Di, and the guidance parameter bw is dynamically adjusted to avoid blind adjustment of the adjustment step length in the search, thereby effectively adjusting the structure of the solution. The improvisation process of the ANGHS algorithm is as follows: end ) ( () else end end () 2 end HM HM HM else )/1000 LB -(UB 0 HM PAR rand() ) ( () / * update /* HMCR rand() do N 1,2,.., i ' ' ' ' ' ' ' best worst i best i i i best ' end LB UB rand LB x LB x LB x elseif UB x UB x if bw bw rand x x HM bw bw HM if if x x rand x x positon x x F x if f or i i i i i i i i i i i i i i worst worst worst i R worst i i worst i best i R                             

Adjustment Parameters F

Adjustment parameter F is defined as formula (10):

*(F F )

MAX MAX MIN

FF rand  ₍₁₀₎

where, FMIN 1;FMAX 2_, rand _{is a random number between (0,1). In the search process, the}

Harmony Memory Diversity

The harmony memory diversity refers to the difference between the optimal harmony vector and the worst harmony vector in the harmony memory [15]. The greater the difference, the better the harmony memory diversity and the smaller the difference, indicating that the diversity of the harmony memory is worse. In order to specifically represent this concept, we introduce the vector norm to define the diversity of the harmony memory. The definition is as formula (11):

best worst i

D  HM HM

(11) where, Di is the diversity of the i-th harmony memory, HMbest and HMworst respectively

represent the optimal and worst harmony vectors in the i-generation harmony memory.  represents the 1 norm of a vector.

Parameter bw Adjustment

In the algorithm search process, the adjustment of the parameter bw plays an important role in the optimization of the entire algorithm, but the adjustment of the parameter bw is difficult to control. Therefore, in this algorithm, we adjust the dynamic parameters bw according to the diversity of the harmony memory. The specific adjustment methods is as Equation(12):

(UB LB ) / 1000 , 0

, otherwise

i i i

best worst

i i

i

D

bw HM HM

D

 

 

 



 ₍₁₂₎

where UBi and LBi represent the upper and lower bounds of the i-th dimension vector, Di is the

diversity of the i-th harmony memory, HMbest and HMworst respectively represent the optimal and worst harmony vectors in the i-generation harmony memory.

Position Update Strategy

In the ANGHS algorithm,

worst i x

symmetrical point xR _about best i x

is generated by adjusting the parameter F as Equation (13):

best worst

R i i

x  F x x

(13) where, F is the adjustment parameter. The position update operation is inspired by the idea of particle swarm algorithm, the worst harmony always approaches the best harmony, in the search process, the harmony vector difference in the entire harmony memory becomes smaller and smaller, and the diversity of the harmony memory deteriorates, and it is easy to fall into a local optimum.

[image:5.595.51.549.87.241.2]

Table 1. Seven Benchmark functions.

Function Expression Search range Optimal value

sphere [-100,100] 0

Rotated [-100,100] 0

Rosenbrock [-30,30] 0

Rastrigin [-5.12,5.12] 0

Griewank [-600,600] 0

Ackley [-32,32] 0

[image:5.595.66.534.280.394.2]

Schwefel’s Problem1.2 [-100,100] -450

Table 2. Five different algorithm parameter settings.

Algorithm HMS HMCR PAR bw NI HS 5 0.99 0.3 0.01 10000 IHS 5 0.99 PARmax=0.99

PARmin=0.01

bwmax=(Ui-Li)/20 bwmin=0.0001

10000

DSHS 5 0.99 PARmax=0.99 PARmin=0.03

0.001 10000

NGHS 5 - PARm=0.005 - 10000 PSONHS 5 0.99 0.1 - 10000

Table 3. Optimization results of benchmark functions.

Function Index HS IHS DSHS NGHS ANGHS

sphere Best Worst Mean Std 0 6.6583E+04 2.2194E+03 1.2156E+04 0.3703 3.5131 1.4106 0.6889 0 6.1169E+03 203.8983 1.1168E+03 0 8.6488E+04 2.8829E+04 1.5790E+04 3.4703E-06 1.7235E-05 7.9228E-06 3.9034E-06 Rotated Best Worst Mean Std 0 3.4787E+07 1.1596E+06 6.3511E+06 6.6253E+05 1.6807E+06 1.0809E+06 2.9492E+05 0 6.2823E+05 2.0941E+04 1.1470E+05 0 5.5321E+07 1.8440E+06 1.0100E+07 6.3560E-04 0.0060 0.0030 0.0018 Rosenbrock Best Worst Mean Std 0 3.8792E+06 1.2931E+05 7.0825E+05 18.0761 579.8537 138.7775 108.0987 0 6.2344E+03 207.8143 1.1382E+03 0 3.0763E+06 1.0254E+05 5.6165E+05 28.1008 28.1008 28.1008 0 Rastrigin Best Worst Mean Std 0 448.7200 14.9573 81.9247 7.8119 15.9389 10.9479 2.1960 0 9.8871 0.3296 1.8051 0 456.1385 15.2046 83.2791 0 2.5716E-05 8.5720E-07 4.6951E-06 Griewank Best Worst Mean Std 0 583.3457 19.4449 106.5039 0.0647 1.0174 0.3907 0.2583 0 51.0517 1.7017 9.3207 0 204.5382 23.4846 128.6305 0 0.8726 0.0684 0.2131 Ackley Best Worst Mean Std 0 20.8541 0.6951 3.8074 0.9008 2.3472 1.7066 0.3493 0 11.8485 0.3949 2.1632 0 21.0018 0.7001 3.8344 0 7.1054E-15 2.4869E-15 1.9006E-15    N i i x x f 1 2 1( )

2

1 1

2( )  ( )

   N i i j j x x f         1 1 2 2 2 1

3( ) (100 ( ) ( 1))

N

i

i i

i x x

x x f       N i i i x x x f 1 2

4( ) ( 10 cos(2π ) 10) 1 ) cos( 4000 1 ) ( 1 1 2

5    

  N i i N i i i x x x f ) 20 ) 2 cos 1 exp( 1 2 . 0 exp( 20 ) ( 1 ` 2

6 x e

N x N x f N I i N i

i  

   _{ }   π 450 ) ) 1 , 0 ( 4 . 0 1 ( ) ( ) ( 2 1 1

7    

[image:5.595.63.528.432.767.2]

Schwefels Problem1.2

Best Worst Mean Std

0

4.6819E+07 1.5606E+06 8.5479E+06

4.2427E+06 1.5644E+07 8.2095E+06 2.7671E+06

0

2.3053E+06 7.6844E+04 4.2089E+05

0

4.7227E+07 1.5742E+06 8.6225E+06

-450 -450 -450 0

Simulation Experiment

Experimental Preparation

In order to evaluate the performance of the Adaptive novel global harmony search algorithm (ANGHS) proposed in this paper, seven complex Benchmark test functions are selected to test and analysis the standard harmony search algorithm (HS), the improved harmony search algorithm (IHS), the Dynamic Self-adaptive Harmony Search algorithm (DSHS), the novel global optimal harmony search algorithm (NGHS), and the Adaptive novel global harmony search algorithm (ANGHS) proposed in this paper. In the experiments, seven Benchmark functions are shown in Table 1. The parameter settings of the compared HS algorithms are shown in Table 2.

Experimental Results and Analysis

In order to ensure the fairness of the algorithm comparison, the maximum number of evaluations of the objective function by each algorithm is used as the iterative termination condition. The maximum number of iterations of the algorithm J is taken as 10000, each algorithm runs independently 30 times. Use Best for Best Value, Worst for Worst Value, Mean for Average Value, and Std for Variance Value. The test results for the 7 functions are shown in Table 3.

According to Table 3, the ANGHS algorithm obtains the optimal solution for the best values, the worst values, the average values, and the variances of the functions Rastrigin, Griewank, Ackley and Schwefels Problem1.2. This shows that the ANGHS algorithm is well optimized for the above four functions in the optimization accuracy, the search speed of the algorithm, the average convergence speed and the stability. For the sphere and Rotated functions, the worst value, mean value and variance of the ANGHS algorithm all get the best results, although Best does not get the optimal solution. However, it is very close to the optimal solution. For the Rosenbrock function, the ANGHS algorithm obtains the optimal solution in addition to the best value. This shows that the algorithm has achieved good results in the convergence speed and global stability of the three functions. The HS, DSHS, and NGHS algorithms are for the sphere, Rotated, Rosenbrock, Rastrigin, Griewank and Ackley functions, each time the Best value can get the optimal solution, indicating that these three algorithms have strong local search ability and high search precision. In general, ANGHS algorithm achieves better performance in terms of convergence speed and stability than HS algorithm, IHS algorithm, DSHS algorithm and NGHS algorithm.

Conclusions

The ANGHS algorithm proposed in this paper aims at the premature convergence of the NGHS algorithm, and it is easy to fall into the local optimal problem to improve. The adjustment parameter F is introduced to generate a new location updating strategy, and the parameter BW is adjusted using the diversity of the harmony memory to improve the local search ability of the algorithm in the solution space so that the algorithm has good convergence and prevents the algorithm from prematurely converging. The AGHS algorithm is compared with the HS, IHS, DSHS, NGHS algorithm through six test functions. The simulation results show that the AGHS algorithm shows better performance in average search speed, convergence speed and stability.

Acknowledgment

References

1. Zou D, Gao L, Li S, et al. An effective global harmony search algorithm for reliability problems, Expert Systems with Applications, 2011, 38(4): 4642-4648.

2. Shou-heng Tuo, Fang-an Deng. Dynamic Self-adaptive Harmony Search Algorithm for Solving High-dimensional Complex Optimization Problems, Computer Science, 2012, 39(9):240-246.

3. Geem Z W, Kim J H, Loganathan G V.A new heuristic optimization algorithm: Harmony search, Simulation, 2001, 76(2):60-68.

4. Yuan-yuan Zhou, Xian-de Hu, Jing-ming Li. An Improved Global Harmony Search Algorithm for Solving Function Optimization Problems, Journal of Chifeng University (Natural Science Edition), 2016, 32(11): 15-17.

5. Mahdavi M, Fesanghary M, Damangir E. An improved harmony search algorithm for s-olving optimization problems, Applied Mathematics and Computation, 2007,188(2):156-157.

6. Mahdavi M, Global-best harmony search. Applied Mathematics and Computation 2008,198(2):643-656.

7. Xiang W, An M, Li Y, et al. An improved global-best harmony search algorithm for faster optimization, Expert Systems with Applications, 2014, 41(13): 5788-5803.

8. Mohammed El-Abd. An improved global-best harmony search algorithm, Applied Mathematics & Computation, 2013, 222(5):94-106.

9. Hui-hui Yan, Jun-hua Duan, Biao Zhang. Harmony Search Algorithm With Self-adaptive Dynamic Parameters, Control & Decision Conference, 2015,1221-1226.

10. Pan QK, Suganthan PN, Tasgetiren MF, et al. A self-adaptive global best harmony search algorithm for continuous optimization problems, Applied Mathematics & Computation, 2010, 216(3): 830-848.

11. Zou D, Gao L, Wu J, et al. A novel global harmony search algorithm for reliability problems. Computers & Industrial Engineering, 2010,58(2): 307-316.

12. Jing Chen, Quan-ke Pan, Jun-qing Li. Harmony search algorithm with dynamic control parameters. Applied Mathematics & Computation, 2012, 219(2):592-604.

13. Jun-chang Zhai, Li-Qun Gao, Hai-Bin Ouyang, et al. Improved Novel Global Harmony Search Algorithm. Journal of Northeastern University (Natural Science) ,2015,36(10):1403-1437.

14. M. Mahdavi, M. Fesanghary, E. Damangir. An improved harmony search algorithm for solving optimization problems, Applied Mathematics and Computation, 2007, 188 (2):1567–1579.