2016 International Conference on Artificial Intelligence: Techniques and Applications (AITA 2016) ISBN: 978-1-60595-389-2
Extreme Learning Machine Optimized by Improved Firefly Algorithm
Ze-kun ZHOU
*and Bin JIAO
Shanghai Dianji University, Shanghai, China
*Corresponding author
Keywords: Firefly algorithm, Function optimization, Extreme learning machine.
Abstract. As a simple and effective feedforward neural network, extreme learning machine (ELM) can randomly generate the connection weight between input layer and hidden layer and the hidden layer neuron threshold. Extreme learning machine can be used to solve the classification problem, but its classification accuracy is not good enough. In this paper, we proposed an improved firefly algorithm called IFA and use it to select the parameters in ELM. Experimental results showed that the IFA can solve the premature problem and the classification ability of ELM can be improved by the use of IFA.
Introduction
The firefly algorithm is a kind of swarm stochastic optimization algorithm, which is based on the characteristics of the light emitting and aggregation behaviors of the firefly in the natural world [1].
In the running process, firefly use their own fluorescein to forage food and communicate with other fireflies around. In general, the higher the individual fluorescein, the more fireflies will gather around. Firefly move through searching other individual, which is more bright.
But the basic firefly algorithm has the disadvantage about falling into local optimal solution. In order to solve this problem, this paper proposed an improved firefly algorithm by changing the displacement formula. The improved formula took the population information into account and the experimental results showed that the improved firefly algorithm can solve the premature problem and can be used in function optimization [2].
The feedforward neural network with gradient descent has disadvantage about slow training speed and being easy to fall into local optimal solution. Therefore, researchers have put forward the theory about extreme learning machine. The theory is a simple and effective method of single hidden layer feedforward neural network, which can randomly generate the connection weight ‘w’ between input layer and hidden layer and the hidden layer neuron threshold ‘b’ [3]. In addition, for these two randomly selected parameters, there is no need to adjust them in ELM training process. By setting the number of neurons in the hidden layer, we can get the only optimal solution. Compared with the previous neural network method, ELM has the advantages about fast learning speed, good generalization performance, and so on [4].
Based on the above advantages, ELM can be used in many classification problems. But the classification accuracy of basic ELM is not good enough. In this paper, the improved firefly algorithm is used to select the parameters in ELM so as to improve its classification ability. After establishing the IFA-ELM model, the UCI data set is used to verity the classification accuracy of the model.
IFA Introduction
Basic Firefly Algorithm
1) The fluorescence intensity of firefly can be defined as follows:
ij
r
e I I 0
(1) In the above formula, I0 is the fluorescence intensity of firefly’s current position; is absorption coefficient; rij is the spatial distance between the ith firefly and the jth firefly.
The attraction of firefly can be defined as follows:
2 0 ij r e (2) In the above formula, 0 is the maximum fitness factor.
The displacement of firefly can be defined as follows:
)2 1 (
1
rand x
x x
x j i
k i k
i
(3) In the above formula, xi and xj is the ith and jth firefly’s position and the jth firefly is more bright
than ith firefly. is step length factor; ) 2 1 (rand
is the random disturbance term.
Improved Firefly Algorithm
In order to solve the problem that basic firefly algorithm is easy to fall into local optimal solution, we proposed IFA model. In this model, we put the brightest individual into account. So, fireflies can learn the best individual position in the process of moving. This method can increase the global search speed. In addition, in order to make fireflies move reasonably, several weight factors were set up to control them. On the other hand, we improve the fixed step length factor. The new step length factor will decrease linearly based on iteration number. This approach can make fireflies have higher global search ability in the initial iteration and have higher local search ability in later iteration.
The improved formula shows as follows:
) 2 1 ( 3 2 11
rand x x w x x w x w
x best i j i
k i k
i
(4) max / 1 T x xbest i
e
w (5)
In the above formula, xbest is the best firefly’s position.
exp10 1 / 1 1 / exp10 1
5 . 0 4 .
0 max max
2 T t T
w
(6)
exp10 1/ 1 1/ exp10 1
5 . 0 4 . 0 max
3 t T
w
(7)
max
/ max6 . 0 2 .
0 T t T
(8)
In the above formula, Tmax is the maximum number of iteration; t is the current iteration number.
Numerical Simulation
Table 1. Standard testing functions.
F Expression Search space
f1 f(x,y)20(x210cos(2x)y210cos(2y)) (-5.12,5.12)
f2 f(x,y)x2y2 (-10,10)
f3
n
i i
ix x
f
1 2
)
( (-10,10)
f4
n
i
i i
i x x
x x
f
1
2 2
) 2 sin 1 . 0 2 . 0 ( )
( (-5.12,5.12)
f5
n
i i
x x
f
1
2
) 5 . 0 ( )
( (-10,10)
In these 5 test functions, their theoretical optimal solutions are zero. In order to illustrate the contrast result, basic firefly algorithm and the method used in literature [6] are used to compare with IFA. In algorithm setting, the population is 20, the iteration number is 200. The maximum value, minimum value and average value are used to evaluate the performance of these algorithm. The experimental result shows as follows:
Table 2. Function optimization results.
F Index FA Method in literature [6] IFA
f1
Max 47.451241135663125 52.428800000000003 39.310308038130714
Min 1.704010046486241e-05 6.296516820038432e-05 5.839177748686377e-08
Avg 55.341712662717001 47.491302502437605 0.432012699751643
f2
Max 2.214245507166926e+02 2.210679248223509e+02 1.484506998929774e+02
Min 17.869664095600982 4.088841158740642 4.225244872202438e-05
Avg 88.755154694466086 73.787468120672258 21.086184303989729
f3
Max 4.988749994163458e+02 5.778908063965245e+02 4.229468009706569e+02
Min 11.451273687361176 2.179979103941653 1.099418340041414e-06
Avg 1.650969150893980e+02 1.208128857465040e+02 1.814411620536711
f4
Max 15.056522759951639 19.080158331670273 11.155926707265529
Min 5.091312900885789 0.385135717584443 1.483206180031726e-08
Avg 20.696190725832743 14.182325271070468 0.136922316543382
f5
Max 3.074637736838682e+02 3.307500000000000e+02 2.647922212506767e+02
Min 10.483557530507118 3.298820675640287 4.983022936956864e-05
Avg 97.264123893233162 78.942514915631492 1.703633081804124
As we can see in the above table, the IFA algorithm in this paper did better than other two models in function optimization. It has a fast convergence speed and solves the premature problem.
IFA-ELM Model
n l l
l
n n
w
ln 2
1
2 22
21
1 12
11
(9) In basic ELM, the connection weight ‘w’ between input layer and hidden layer and the hidden layer neuron threshold ‘b’ are selected randomly. In order to improve the classification ability of ELM, IFA is used to select these two parameters. Use classification accuracy of the training sample as a function fitness. Firefly will continue to search the optimal parameters by iteration.
Simulation Verification
In this section, two UCI data set is chosen to verify the classification ability of IFA-ELM model. In these two data sets, iris is a three classification problem, it contains 150 samples and 4 attributes; wine is a three classification problem, it contains 178 samples and 14 attributes. Considering the randomness of the data extraction, the randperm function is used to create random numbers for selecting train samples and test samples. In order to illustrate the classification performance, basic ELM model and FA-ELM model are used to compare with the IFA-ELM model. The experimental results are as follows:
Table 3. UCI standard data sets.
Data sets Training samples’ number Testing samples’ number
Wine 128 50
[image:4.595.57.540.354.469.2]Iris 100 50
Table 4. The comparison of classification accuracy.
Data sets ELM FA-ELM IFA-ELM
Wine 93.2% 94.8% 96.8%
Iris 94% 95.6% 98.4%
[image:4.595.76.514.508.684.2]For further illustrate the classification results, the experimental result figures are show as follows:
Figure 1. IFA-ELM classification results.
Conclusion
solve the premature problem. On the other hand, UCI data set are used to verify the effectiveness of IFA-ELM model. Result showed that IFA-ELM has a good performance in classification.
References
[1] Yang, Xin‐She. Firefly Algorithm[M]// Engineering Optimization. 2010:221-230.
[2] Yang X S. Firefly Algorithm, Lévy Flights and Global Optimization[M]// Research and Development in Intelligent Systems XXVI. Springer London, 2010:209-218.
[3] Huang G B, Zhu Q Y, Siew C K. Extreme learning machine: Theory and applications[J]. Neurocomputing, 2006, 70(1-3):489-501.
[4] Huang G B, Zhou H, Ding X, et al. Extreme learning machine for regression and multiclass classification.[J]. IEEE Transactions on Systems Man & Cybernetics Part B Cybernetics A Publication of the IEEE Systems Man & Cybernetics Society, 2012, 42(2):513-29.
[5] Senthilnath J, Omkar S N, Mani V. Clustering using firefly algorithm: Performance study[J]. Swarm & Evolutionary Computation, 2011, 1(3):164-171.
[6] Fu Qiang, Tong Nan, Zhao Yiming. Firefly optimization algorithm based on multi group studying mechanism[j]. Computer application research, 2013, 30(12):3600-3602
[7] Huang G B, Ding X, Zhou H. Optimization method based extreme learning machine for classification [J]. Neurocomputing, 2010, 74(1-3):155-163.
[8] Wang H, Wang W, Sun H, et al. Firefly algorithm with random attraction[J]. International Journal of Bio-Inspired Computation, 2016, 8(1):33-41.
[9] Li X, Xie H, Wang R, et al. Empirical analysis: stock market prediction via extreme learning machine[J]. Neural Computing & Applications, 2016, 27(1):67-78.
