! ∀#∃
% &∋& ()∗∃+∀∋,)−∗(.−−(+
/0 % #11& &211
Free Search Applied to Large Constraint
Optimisation Problem
Kalin Penev
Southampton Solent University, Southhampton, UK [email protected]
Abstract. The article presents experimental results achieved by Free Search on optimisation of 100 dimensional version of so called bump test problem. Free Search is adaptive heuristic algorithm. It operates on a set of solutions called population and it can be classified as population-based method. It gradually modifies a set of solutions according to the prior defined objective function. The aim of the study is to identify how Free Search can diverge from one starting location in the middle of the search space in comparison to start from random locations in the middle of the search space and start from stochastic locations uniformly generated within the whole search space. The results achieved form the experiments with above initialisation strategies are presented. A discussion focuses on the ability of Free Search to diverge from one location if the process stagnates in local trap during the search. The article presents, also, the values of the variables for the best achieved results, which could be used for comparison to other methods and further investigation.
1
Introduction
results achieved by Free Search are for 20 dimensions 0.80361910412558 and for 50 dimensions 0.83526234835811175. This investigation has no available publi-cations for other algorithms for 100 dimensional variant of the bum problem. The search space of this test is continuous. It has many peaks. An essential con-dition of this test is start from single location in the middle of the search space. This condition guarantees start from location, which is relatively far from the maximal hill. In contrast to start from multiple start locations uniformly dis-tributed within the search space, it eliminates possibility for starting from initial locations accidentally generated near to the best value. Start from one location facilitates a measurement of the divergence across the whole space and then con-vergence to the best value. The work presented in this article is a continuation of the experiments on this problem published earlier [11, 17]. The experiments with the bump optimisation problem for n = 50, xi ∈ (0,10), i = 1, . . . ,50
require, for clarification of the results with precision of seven decimal digits, exploration of 10400
solutions ... Free Search achieves the maximum with such precision after exploration of less than 109
solutions... The relation between the number of all possible values with a certain level of precision and the number of explored locations deserves attention. For 50-dimensional space the relation is 109
/10400
∼1/1044 . These results can be considered from two points of view.
The first point of view is when these results are accepted as accidental. The algorithm is “lucky” to “guess” the results. If that point of view is accepted, it follows that the algorithm “guesses” the appropriate solution from 1044
possible for 50-dimensional space. ...Another point of view is to accept the results as an outcome of the intelligent behaviour modeled by the algorithm. Free Search ab-stracts from the search space essential knowledge. That knowledge leads to the particular behaviour and adaptation to the problem. In that case the relation between explored and all possible locations can be a quantitative measure of the level of abstraction. The second point of view that the algorithm models intelli-gent behaviour is accepted. Abstraction knowledge from the explored data space; learning, implemented as an improvement of the sensibility; and then individual decision-making for action, implemented as selection of the area for next explo-ration; can be considered as a model of artificial thinking [11]. The experiments with 100 dimensional variant in this study are harder and Free Search confirms its excellent exploration abilities on large-scale optimisation problems. In the article and in the tables the following notation is accepted:n is the number of dimensions.i is dimensions indicator, i= 1, . . . n.xi are initial start locations.
XmaxandXmin are search space boundaries.xmaxare variables of an achieved
local maximum.Fmax100is a maximal achieved value for the objective function.
ri is random value,ri∈(0,1).
2
Test Problem
Maximise:f(xi) = n X i=1 cos4
(xi)−2 n Y
i=1
cos2 (xi)
/ v u u t n X i=1 ix2 i subject to: n Y i=1
xi>0.75 (1)
and
n X
i=1
xi<15
n
2 (2)
for 0< xi<10, andi= 1, . . . n,, starting fromxi= 5,i= 1, . . . , n, wherexi
are the variables (expressed in radians) and n is the number of dimensions [7].
3
Methodology
Free Search is applied to the 100 dimensional variant of the bump problem as follow: The population size is 10 (ten) individuals for all experiments. Four series of 320 experiments with four different start conditions are made:
– start from one location in the middle of the search spacexi = 5,i= 1, . . . , n;
(This is an original condition of the test, ignored from majority authors due to inability of other methods to diverge successfully from one location [8, 9, 18]
– start from random locations in the middle of the search spacexi= 4 + 2∗ri,
ri∈(0,1);
– start from locations stochastically generated within the whole search space
xi= (Xmax−Xmin)∗ri,ri∈(0,1);
– additional experiments with start from the best achieved locationxi=Xmax
are made. The last result of these experiments is presented also in Table 1.
For the first three series maximal neighbour space per iteration is restricted to 5% of the whole search space and sensibility is enhanced to 99.999% from the maximal. For the fourth series, in order to distinguish very near locations with very similar high quality, maximal neighbour space per iteration is restricted to 0.0005% from the whole search space and sensibility is enhanced to 99.9999999% from the maximal.
4
Experimental Results
The maximal values achieved from 320 experiments for 100-dimensional search space for the four different start conditions are presented in Table 1. In Table 1:
xi = 5 start from one location;xi= 4 +ri start from random locations in the
Table 1.Maximal values achieved for n = 100 bump problem
n start from iterations constraint (1)>0.75 Fmax100
100 xi= 5 100000 0.750305761739250 0.838376006873 100 xi= 4 +ri 100000 0.7750016251034640 0.836919742344 100 xi= (Xmax−Xmin)∗ri 100000 0.750125157302112 0.838987184912 100 xj=Xmax 100000 0.750000000039071 0.845685456012
uniformly distributed within the whole search space; xi =Xmax start from a
local sub-optimum.
The best results from 320 experiments with start from currently achieved best local sub-optimumxi=Xmaxare presented in Table 2. The variables’ values for
the best achieved objective function value forn= 100 are presented respectively in Table 3. The constraint parameter values are an indicator whether the found maximum belongs to the feasible region. They indicate also expected possible improvement and can be valuable for further research.
The best results presented on the Table 2 suggest that for 100-dimensional search space n = 100 with the precision of four decimal digits (0.0001) the optimum isFopt100= 0.8456. Considering constraint value with high probability could be accepted that the optimal value for 100-dimensional variant of the bump problem is between 0.845685 and 0.845686. Clarification of this result could be a subject of future research. From previous experiments Free Search achieves:”For
n = 50, xi ∈ (0,10), i = 1, . . . ,50 there are 10 400
solutions with a precision of seven decimal digits. Free Search achieves the maximum with such precision after exploration of less than 109
solutions” [11].
For n= 100, xi ∈ (0,10), i = 1, . . . ,100 with a precision of three decimal
digits there are 10400
solutions. To reach the result with this precision current version of Free Search needs exploration of more than 1012
solutions. These results suggest that perhaps 100-dimensional space is more complex than 50-dimensional space with the same size.
Let us note that: (1) these results are achieved on a probabilistic principle;(2) the search space is continuous and the results can be clarified to an arbitrary precision; (3) precision could be restricted from the hardware platform but not from the algorithm [15, 16] .
5
Discussion
Table 2.Best achieved results from 320 experiments start fromxi=Xmax Fmax= 0.84568545600962819 px= 0.7500000000777115
Fmax= 0.8456854560029361 px= 0.75000000002732459
Fmax= 0.84568545601228962 px= 0.75000000003907141
Fmax= 0.84568545600160894 px= 0.75000000003325862
Fmax= 0.84568545601179412 px= 0.7500000000164837
Fmax= 0.84568545600045464 px= 0.7500000000902789
Fmax= 0.84568545600126788 px= 0.75000000002984646
Fmax= 0.84568545600471334 px= 0.75000000013973589
Fmax= 0.84568545600142975 px= 0.75000000000654199
Fmax= 0.84568545600266842 px= 0.75000000002129508
Fmax= 0.84568545600390022 px= 0.75000000011346923
Fmax= 0.84568545600150213 px= 0.75000000021696134
Fmax= 0.84568545600245093 px= 0.75000000004234102
Fmax= 0.84568545600018008 px= 0.75000000002819023
Fmax= 0.84568545600770795 px= 0.75000000006311085
Fmax= 0.84568545600403033 px= 0.75000000001039713
Fmax= 0.84568545600069034 px= 0.75000000001963218
Table 3.Variables values for the best achieved Fmax100=0.84568545601228962
x[0]=9.4220107126347745 x[1]=6.2826322016103955 x[2]=6.2683831748189975
x[3]=3.1685544118834987 x[4]=3.1614825164328604 x[5]=3.1544574053305716
x[6]=3.1474495832290019 x[7]=3.1404950479045355 x[8]=3.1335624853236599
x[9]=3.1266689747394079 x[10]=3.1197805974560233 x[11]=3.1129253611190442
x[12]=3.1061003015872641 x[13]=3.0992700218084379 x[14]=3.0924554966175832
x[15]=3.0856546345558589 x[16]=3.0788589079539115 x[17]=3.0720627263662514
x[18]=3.0652657000077048 x[19]=3.0584746707061194 x[20]=3.0516637034865339
x[21]=3.0448584949106863 x[22]=3.0380286236964169 x[23]=3.0311906892219178
x[24]=3.024325642570814 x[25]=3.0174564447484493 x[26]=3.0105540636060151
x[27]=3.0036206348205221 x[28]=2.996654984477289 x[29]=2.9896634779600619
x[30]=2.9612192303388043 x[31]=2.9755508892857203 x[32]=2.9684029022824512
x[33]=2.9612192303388043 x[34]=2.9539969359393266 x[35]=2.9467002904105652
x[36]=2.9393411868452524 x[37]=2.9319091645017501 x[38]=2.9243754870396326
x[39]=2.916779048836506 x[40]=0.48215961508911009 x[41]=0.48103824318067195
x[42]=0.47987816774664849 x[43]=0.47878167955313988 x[44]=0.47768451249416577
x[45]=0.47661282330477983 x[46]=0.47553403486022883 x[47]=0.47446785125492774
x[48]=0.47342756022045934 x[49]=0.47239007924579712 x[50]=0.47137147513069294
x[51]=0.47032878010013335 x[52]=0.46930402745591204 x[53]=0.46831098721684394
x[54]=0.46735104878920491 x[55]=0.46636407600760849 x[56]=0.46539891729486565
x[57]=0.46441864961851576 x[58]=0.46347276946009547 x[59]=0.46251323773794134
x[60]=0.4616002801440135 x[61]=0.46065790486354485 x[62]=0.45975509243021634
x[63]=0.45882641352502623 x[64]=0.45794463889695297 x[65]=0.45703335381561577
x[66]=0.45616083797292917 x[67]=0.45530545866090766 x[68]=0.4544181045335004
x[69]=0.45356136754388732 x[70]=0.45270186592373279 x[71]=0.45185207109498432
x[72]=0.45101805630688263 x[73]=0.45019912098968307 x[74]=0.44936498811072595
x[75]=0.44854068486603355 x[76]=0.44772742155880246 x[77]=0.44690550854857802
x[78]=0.44610180301058927 x[79]=0.44532932347961096 x[80]=0.44452998407145683
x[81]=0.4437461852789148 x[82]=0.44294433416028023 x[83]=0.44217581932603778
x[84]=0.44144485059511346 x[85]=0.44065197401737571 x[86]=0.43992044125167573
x[87]=0.43915615308675215 x[88]=0.43841742447730969 x[89]=0.43766920812505228
x[90]=0.43695640149660347 x[91]=0.43620589323988396 x[92]=0.43550809702824705
x[93]=0.43477896755895717 x[94]=0.43406590088092134 x[95]=0.43335315514811895
x[96]=0.43265970793420877 x[97]=0.43194897101473584 x[98]=0.43126789176092778
Another aspect of the results is what is the overall computational cost for exploration of this multidimensional task. A product between the number of iterations and the number of individuals in the algorithm population could be considered as a good quantitative measure for overall computational cost. For all experiments population size is 10 (then) individuals. In comparison to other publications [7–9] this is low number of individuals, which lead to low computa-tional cost. To guarantee diversification and high probability of variation of the dimensions values within the population other methods operate on higher num-ber of individuals, pay high computational cost and require large or distributed computational systems and extensive redundant calculations [5, 18]. For his task Free Search minimises required computational resources to a single processor PC.
6
Conclusion
In summary Free Search demonstrates good exploration abilities on 100-dimensional variant of he bump problem. Implemented novel concepts lead to an excellent performance. The results suggest that: (1) FS is highly independent from the ini-tial population; (2) the individuals in FS adapt effectively their behaviour during the optimisation process taking into account the constraints on the search space, (3) FS requires low computational resources and pays low computational cost keeping better exploration and search abilities than the methods tested with the bump problem and discussed in the literature [2, 5, 7, 9, 18] (4) FS can be reliable in solving real-world non-linear constraint optimisation problems. The results achieved on the bump optimisation problem illustrate the ability of Free Search for unlimited exploration. With Free Search, clarification of the desired results can continue until reaching an acceptable level of precision. Therefore, Free Search can contribute to the investigation of continuous, large (or hypothet-ically infinite), constrained search tasks. A capability for orientation and opera-tion within multidimensional search space can contribute also to the studying of multidimensional spaces and to a better understanding of real space. Presented experimental results can be valuable for evaluation of other methods. The algo-rithm is a contribution to the research efforts in the domain of population-based search methods, and can contribute, also, in general to the Computer Science in exploration and investigation of large search and optimisation problems.
References
1. Eiben, A.E., and J.E. Smith, Introduction to Evolutionary Computing, Springer, ISBN 3-540-40184-9,(2003), 15–35.
2. EI-Beltagy M.A., and A.I. Keane, Optimisation for Multilevel Problems: A Com-parison of Various Algorithms, In I.C. Parmee editor, Adaptive computing in design and manufacture, ISBN 3-540-76254-X Springer - Verlag London Limited, (1998), 111–120.
4. Holland J., Adaptation In Natural and Artificial Systems, University of Michigan Press,(1975).
5. Jiandong Li, Zhuo Kang, Yan Li, Hongqing Cao, Pu Liu,Automatic Data Mining by Asynchronous Parallel Evolutionary Algorithms, tools, p. 0099, 39th International Conference and Exhibition on Technology of Object-Oriented Languages and Systems (TOOLS39), (2001).
6. Keane A. J.,Genetic algorithm optimization of multipeak problems: studies in con-vergence and robustness, Artificial Intelligence in Engineering 9(2), (1995), 75–83. 7. Keane A. J., A Brief Comparison of Some Evolutionary Optimization Methods, In
V.Rayward-Smith, I. Osman, C. Reeves and G.D. Smith, J. Wiley (Editors), Modern Heuristic Search Methods, ISBN 0471962805, (1996),255–272.
8. Michalewicz, Z. and Schoenauer, M.,Evolutionary Algorithms for Constrained Pa-rameter Optimization Problems, Evolutionary Computation, Vol.4, No.1, (1996),1– 32.
9. Michalewicz, Z. and Fogel, D., How to Solve It: Modern Heuristics, ISBN 3-540-66061-5 Springer-Verlag, Berlin, Heidelberg, New York, (2002).
10. Penev, K., and Littlefair, G.,Free Search - a Novel Heuristic Method, Proceedings of PREP 2003, 14-16 April, Exeter, UK, (2003) 133–134.
11. Penev K.,Adaptive Computing in Support of Traffic Management, In I.C. Parmee editor, Adaptive Computing in Design and Manufacturing, ISBN 1852338296, (ACDM 2004), 20- 22 April, 2004, Bristol, UK, (2004), 295–306.
12. Penev K., and Littlefair G., Free Search A Comparative Analysis, Information Sciences Journal, Elsevier, Volume 172, Issues 1-2, (2005), 173–193.
13. Penev, K., Adaptive Search Heuristics Applied to Numerical optimisation,Thesis for a degree of Doctor of Philosophy, Nottingham Trent University, UK, (2005). 14. Penev, K.,Heuristic Optimisation of Numerical Functions, 20th International
Con-ference Systems for Automation of Engineering and Research, ISBN-10: 954-438-575-4, September, Varna, Bulgaria, (2006), 124–129.
15. Penev, K.,Novel Adaptive Heuristic for Search and Optimisation, in Proceedings of IEEE John Vincent Atanasoff International Symposium on Modern Computing, 3-6 October, Sofia, Bulgaria, ISBN-10: 978-0-7695-2643-0, (2006), 149–154.
16. Penev, K., Free Search Towards Multidimensional Optimisation Problems, in Guerrero-Bote, V. P. editor, Current Research In Information Science And Tech-nologies, Multidisciplinary Approaches To Global Information Systems, Volume II, ISBN-10: 84-611-3105-3, October, 2006, Spain, (2006), 233–237.
17. Penev, K., Free Search and Differential Evolution Towards Dimensions Number Change, International Conference Artificial Neural Networks and Intelligent Engi-neering 2006 (ANNIE 2006), 5-8 November 2006, St. Louis, Missouri, USA, (2006). 18. Schoenauer, M. and Michalewicz, Z., Evolutionary Computation at the Edge of
