Particle Swarm Optimization Algorithm Based on Semantic Relations and Its Engineering Applications

(1)

Systems Engineering Procedia 5 ( 2012 ) 222 – 227

Particle swarm optimization algorithm based on semantic relations

and its engineering applications

Liangshan Shao

1

, Yuan Bai

1

, Yunfei Qiu

1

, Zhanwei Du

2 a_{College of Science of Liaoning Technical University, Liaoning, China,}

b_{College of Computer Science and Technology, Jilin University, Changchun, China}

Abstract

Particle swarm optimization algorithm (PSO) is a good method to solve complex multi-stage decision problems. But this algorithm is easy to fall into the local minimum points and has slow convergence speed, According to the semantic relations, an improved PSO algorithm has been proposed in this paper. In contrast with the traditional algorithm, the improved algorithm is added with a new operator to update its crucial parameters. The new operator is to find out the potential semantic relations behind the history information based on the ontology technology. Particle swarm optimization can be applied to many engineering fields, taking Traveling Salesman Problem (TSP) as example. Our experiments show accuracy of the improved particle swarm algorithm that is superior to that obtained by the other classical versions, and better than the results achieved by the compared algorithms, besides, this improved algorithm can also improve the searching efficiency.

engineering;TSP

1. Introduction

In the present days, some technologies are highly parallel, self-organized, and full of vigor and vitality in computation intelligence, such as neural networks, cell automation, and so on. Particle swarm optimization (PSO) is originally introduced by Kennedy and Eberhart [1] inspired by stochastic population. This algorithm is motivated by intelligent collective behavior of flocks of birds to decide its searching direction and searching velocity. The most important advantages of the PSO, compared to other optimization strategies, are that PSO is easy to implement and has few parameters to adjust.

PSO is easy to sink into the local optima when solving the functions of high dimension space and so on. How to solve these problems, therefore, is one of the research hotspot. Shi and Eberhart[2] introduced Inertia weight parameter in an effort to improve the searching capacity and the convergence speed. Kennedy [3] divided the main population into a number of sub-swarms with the hope of exploring different areas of the search space. Clerc[4] demonstrated PSO’s stability and concentration for high dimension space. F.Bergh and A.P. Engelbrecht [5] proposed A New Locally Convergent Particle Swarm Optimiser characterized by the low local convergence. In [6], an adaptive PSO algorithm was proposed to ensure the stability in order to avoid premature convergence. The analysis of PSO’s parameters and its convergence was done in [7].

However, all of these algorithms above just pay most of their attention to the particles and routes but little to the relations behind the behavior of particles. Generally speaking, a better particle has some kind of behavior patterns which are better than the other ones. There will be some amends to PSO’s drawbacks. If we could take the semantic relations into consideration, the potential patterns can be expressed by the semantic relations. However, the impact © 2012 Published by Elsevier Ltd. Selection and peer-review under responsibility of Desheng Dash Wu.

Open access under CC BY-NC-ND license.

(2)

factor was ignored which might lead to another chance to improve the algorithm’s performance.

In this paper, an improved particle swarm optimization algorithm based on semantic relations is presented to solve these drawbacks. During the realization process of algorithm, this algorithm, which considers group behavior and the ontology model, focuses on mining the potential semantic relationships and producing the corresponding semantic relations library. Then according to the semantic relations, the new solution is produced. With applying this improved particle swarm optimization algorithm based on Ontology to solve traveling salesman problem, the experimental results indicate that this algorithm improves the performance when looking for the global advantages.

The remainder of this paper is organized as follows. In section2, the semantic relations operator has been discussed and presented. In section3, the proposed algorithm is showed. And in section4, the experimental results and performance comparison are showed. Finally, Section5 draws the conclusion.

2. The semantic relations operator

2.1. Ontology and semantic relations based on Ontology

The concept of ontology primordially derives of philosophy. It is defined as system description of objective things in the world, namely Ontology, a system explanation or description of objective existence, and it cares about abstract nature objective existence. Studer[8] thinks Ontology is the clear formal specification sharing the conceptual model. This includes conceptual model, clear, formalization and sharing.

Generally speaking, Ontology describes concepts and the relationships between the concepts in many domains with normalization and formalization, so as to achieve the purpose of sharing and provide the unified language for the exchange between heterogeneous systems[9-10].

In order to research how to use ontology to organize knowledge, five basic modeling language are proposed, therefore ontology can be said as O=(C,R,F,A,I), and C, R, F, A and I express concept, relations, functions, justice in ontology respectively. A simple small ontology can be expressed as O: (C, R).

This algorithm guides the mining process by ontology. First of all, it can pretreat input data by domain knowledge, and start semantic mining in the semantic level. Analyze implicit information getting by semantic mining to realize the independent learning and perfect the knowledge structure.

2.2. Semantic relations operator to solving TSP

(1) Algorithm thought and flow profile

Optimization Model: An Optimization Model

( , , )

S

:

f

contains three parts: the solution space of discrete S, the constraint setȚand the objective function

f : S

o

R

. The improved algorithm based on subsequent semantic relations is as follows:

Step 1 convert the input vector s into the categories vector according to Ontology

Translate

C , C ,

₁ ₂

, C

_n

into

1 2 n

city city city

C

, C

,

, C

.

Step 2 find semantic relations according to the categories vector Find

C

_city

C

_city

i

o

j .

Step 3 update the semantic relations set

Among them, the main function of the semantic relations discovery algorithm is to find out the potential valuable relations. Their idea is based on this fact that the meaningful relations of categories vector have some statistic features. Therefore, combined with the otology model, the correlative relations found by data mining techniques could be associated with semantic meaning.

(2) Operator in the application of intelligent algorithm The ontology model based on subsequent semantic relations

Definition 1: The ontology model based on subsequent semantic relations: SS= <CS,RS,FS,AS,IS>,

(3)

while CS is the set of concepts; every city is an independent concept here. RS is the set of relations which includes only subsequent semantic relations for the moment. AS is the set of axioms which includes only transitive relations now. IS shows the set of instances. The date would be added into the set as the algorithm executes.

(3) The improved particle swarm optimization algorithm based on subsequent semantic relations

The PSO[11-13] is not suitable for the TSP problem due to its speed formula, thus swap operator and swap sequence was introduced[14-16] to meet this need. Based on the swap operator and the swap sequence, the semantic swap operator is defined as:

Suppose the solution sequence of the TPS problem for n nodes is

S

a ,i 1,

_i

}

, n

, the subsequent semantic swap operator

SE i ,i

₁ ₂

:

If

SE i ,i

₁ ₂

z

, then swap the sequence of

1

i

a

with

2

i

a

, thus

S

c u

S SE i ,i

₁ ₂

is the new solution after S is dealt with by

SE i ,i

₁ ₂

.

If

SE i ,i

₁ ₂

, then do nothing to the S.

Based on [18], the corresponding swap sequence concepts can be got by us.

According to the subsequent semantic swap operator, speed formula can be redefined as:

'

( (

)

(

))

(1

)(

)

id id id id gd id gd id

V

J D

P

X

E

P

X

J

P

X

(1)

while

D E D E

, ( ,

> @

0,1 )

are both random values,

D

(

P

_id

X

_id

)

means the basic swap sequence

(

P

_id

X

_id

)

is kept by the probability of

D

, the same as

E

(

P

_gd

X

_id

)

. It is manifest that the larger

D

is , the more swap operators are kept by

(

P

_id

X

_id

)

, thus the greater the influence of

P

_id is, the same as

E

,

(

P

_gd

X

_id

)

and

P

_gd .

J

is the probability of the basic swap operator, which decreased as the iteration number goes up. The improved PSO algorithm for TSP can be described as follows:

(1) Initialize the parameters, including the initialize solutions for every particles and so on; (2) If the terminal conditions are met, go to (5);

(3) According to particle’s current position

X

_id,compute the next position ' id

X

, the new solution;

1) Compute the difference A between

P

_id and

X

_id,

A

P

_id

X

_id, while A is the basic swap sequence and

id

P

can be got through A and

X

_id;

2) Compute

B

P

_gd

X

_id, while B is a basic swap sequence; 3) Calculate the speed

' id

V

according to formula 1, and transform

V

_id' to a basic swap sequence. 4) Find the new solution

id

'

id id

X

V

;

5) If there is a better solution, then update

P

_id;

(4) If a better solution is found for the whole swarm, then update

P

_gd;

(5) According to the history state information, find the new semantic relations, and update the semantic relation database; then go to (2).

(6) Output the results.

3. The experiment

Experimental simulation platform includes Windows Server 2003, matlab 2008. This experiment compared the improved particle swarm optimization algorithm with the classical particle swarm optimization algorithm.

The experiment uses the data sets of Oliver30, Burma14 and Att48 in TSPLIB[17-18]. In the experiment, Experimental results are as table 1 and table 2. And figure 1-3 respectively give convergence of the algorithm

(4)

minimum, the horizontal axis in the figure display iteration times and longitudinal axis display total path length. The figure 1 shows that the particle swarm algorithm based on subsequent semantic relations has better performance in the minimum and convergence speed. But the figure 2 and 3 show the result of the improved algorithm which simply depends on subsequent semantic relations is not better along with the increase of data.

Table 1 Test results of ant colony

optimal solution PSO Improved PSO

Minimum Maximum median Minimum Maximum median Burma14 30.8785 31.2269 33.4135 31.2269 31.2269 36.2492 31.2269 Oliver30 423.74 434.8286 502.7821 434.8286 425.4752 466.1045 441.5081 Att48 33523 3.6068e+04 3.8836e+04 3.6644e+04 3.8123e+04 4.1320e+04 3.8123E+04

Figure1 Particle swarm optimization algorithm results in data set of Burma14

Figure2 Particle swarm optimization algorithm results in data set of Oliver30

0 5 10 15 20 25 30 420 430 440 450 460 470 480 490 500 510 Iteration number 0 5 10 15 20 25 30 31 32 33 34 35 36 37

(5)

Figure3 Particle swarm optimization algorithm results in data set of Att48

4. Conclusion

By introducing the semantic relations operator based on ontology, then putting forward the particle swarm optimization algorithm based on semantic relations, and testing on the engineering applications, The TSP experimental results show that: for small data sets, optimization ability of particle swarm optimization algorithm based on the semantic relations is even better than classical algorithms, This algorithm can reduce the number of iterations. The particle swarm optimization algorithm based on the semantic relations of operator not only can find the optimal value in a relatively short period of time, but also can expand the search range, so it can avoid getting into the local advantages and enhance the reliability of the algorithm. To sum up,

According to the existing work, we will have the following issues on further research. 1. Further research the efficiency and convergence of the algorithm.

2. In order to improve optimization ability of the algorithm, look for new semantic relations operator for research large data sets.

Acknowledgements

We thank all the colleagues in our laboratory for their assistance. The work is financially supported by the national natural science funds(70971059), scientific research plan of Liaoning province ˄2010230004˅, Jilin Province Development and Innovation Committee’s High and New Technology Projects (20106421) and Graduate Innovation Fund of Jilin University(20111064).

References

1. Eberhart R,Kennedy J.A new optimizer using particle swarm theory[C]//Proc of 6th International Symposium on Micro Machine and Human Science,Nagoya,Japan. Piscataway NJ˖IEEE Service Center,1995˖39-43.

2. Kennedy J,Eberhart R.A discrete binary version of the particle swarm algorithm[C]//Proc of IEEE Conference on Systems,Man and Cybernetics,1997˖4104-4109.

3. Kennedy, J.: Stereotyping: Improving Particle Swarm Performance with Cluster Analysis. In: Proceedings of the 2000 Congress on Evolutionary Computation, pp. 1507–1512.IEEE, Piscataway (2000)

4. Clerc, M.,Kennedy, J.:The Particle Swarm: Explosion, Stability and Convergence in a Multi-Dimensional Complex Space. IEEE Trans.on Evolutionary Computation 6,58–73(2002)

5. F.V D.Bergh, A.P.Engelberecht. A New Locally Convergent Particle Swarm Optimiser. In:Proceedings of the IEEE International Conference on Systems, Man and Cybernetics,2002,Vol.3,94̚99.

0 5 10 15 20 25 30 3 6 3 7 3 8 3 9 4 4 1 4 2 x 10 4

(6)

6. PAN Feng TU Xuyan, CHEN Jie FU Jiwei, A Harmonious Particle Swarm Optimizer-HPSO[J], Computer Engineering, 2004,6.234-236. 7. CUI Hong- mei, ZHU Qing- bao, Convergence analysis and parameter selection in particle swarm optimization[J], Computer Engineering and Applications, 2007,43(23): 89-92.

8. ZHA Ri-Jun, ZHANG De-Ping, NIE Chang-Hai, XU Bao-Wen, Test Data Generation Algorithms of Combinatorial Testing and Comparison Based on Cross-Entropy and Particle Swarm Optimization Method [J], CHINESE JOURNAL OF COMPUTERS, 2010, 33(10): 1896-1907

9. ArpirezJC, CorchoO, Fernandez-LopezM,Gomez-PerezA. WebODE : A scalable ontological engineering workbench. In: GilY, Musen M, Shavlik J, eds. Proc.of the K-CAP200!.NewYork : ACM Press, 2001, 6-13.

10. GaoShang, Yang Jingyu, Group of intelligent algorithms and its application [M], Water conservancy and hydropower press, 2006 ,1-20. 11. JI Zhen, ZHOU Jia-Rui, LIAO Hui-Lian, WU Qing-Hua, A Novel Intelligent Single Particle Optimizer [J], CHINESE JOURNAL OF COMPUTERS, 2010, 33(3): 556-561.

12. HUANG Han, HAO ZhiFeng, WU ChunGuo, QIN Yong. The Convergence Speed of Particle Swarm Optimization [J], CHINESE JOURNAL OF COMPUTERS,2007, 30(8): 1344-1353.

13. ZHA Ri-Jun, ZHANG De-Ping, NIE Chang-Hai, XU Bao-Wen, Test Data Generation Algorithms of Combinatorial Testing and Comparison Based on Cross-Entropy and Particle Swarm Optimization Method [J], CHINESE JOURNAL OF COMPUTERS, 2010, 33(10): 1896-1907

14. Kuo-Kuang Chu, Chien-I Lee, Rong-Shi Tsai, Ontology technology to assist learners’ navigation in the concept map learning system[J], Expert Systems with Applications, 2010, 38: 11293-11299.

15. Chongchong Zhao, Jing Wang, Wei Hu, Xiao Yu, Xiaofeng Wang, An Ontology-based Semantic Search Model Study[C], 2010 3rd Interational Symposium on Knowledge Acquisition and Modeling, 2010:182-185.

16. HUANG Lan, WANG Kang-ping, ZHOU Chun-guang, PANG Wei, DONG Long-jiang, PENG Li, Particle Swarm Optimization for Traveling Salesman Problems [J], JOURNAL OF JILIN UNIVERSITY (SCIENCE EDITION) 477̚480.

17. Studer, Thomas, Privacy preserving modules for ontologies [C], Lecture Notes in Computer Science, , 2010, 5947: 380-387.

18. HUANG Lan, WANG Kang-ping, ZHOU Chun-guang, PANG Wei, DONG Long-jiang, PENG Li, Particle Swarm Optimization for Traveling Salesman Problems [J], JOURNAL OF JILIN UNIVERSITY (SCIENCE EDITION) 477̚480.

Open access under CC BY-NC-ND license.