Random Search. Statement of the Problem of Optimal Design. Basic Concepts and Definitions

(1)

International Journal of Emerging Technology and Advanced Engineering

Website: www.ijetae.com (ISSN 2250-2459, ISO 9001:2008 Certified Journal, Volume 6, Issue 11, November 2016)

161 Random Search. Statement of the Problem of Optimal

Design. Basic Concepts and Definitions

George Filatov

Professor, Doctor of Techn. Sciences, Ukrainian State University of Chemical Technology, Ukraine

Abstract − This paper is first from the series of articles devoted to application of random search method for the optimal designing of structures operating in a neutral and aggressive media. This article has exploratory in nature and may be useful for young researchers, who begin study the nonlinear methods of search engine optimization. In the article is adduced the setting of search engine optimization tasks, are considered the models of optimized object, are presented the characteristics of the search efficiency, is formulated the idea of learning of random search and self-learning.

Keywords — Random Search, Nonlinear Programming,

Search Engine Optimization, Models of Optimized Object, Search Effectiveness, Self-learning Search Engine

I. INTRODUCTION

Under optimal designing it follows to understand the process of making the best (optimal) in a sense-making with the help of computers. This problem associated with obtaining an optimal solution from the set permissible, is common to all stages of design and largely determines the technical and economic efficiency and manufacturability of designed structures. With the development of computer technology the choice of the optimal solutions in the design stage by means of mathematical modeling is further increased, since virtually eliminated the possibility of experimental optimization of the finished structure.

The common for tasks of the making of optimal decisions that arise at different stages of the design, is that mathematically they can be formulated as a nonlinear programming problem. It is assumed that there is a mathematical model of considered object, which is subjected to optimization and is required for a given model to find parameters that provide extreme value of one of the most important characteristics, provided that the other satisfy the specified constraints system. Towards such formulation we may reduce to the broad class of extreme problems. Including the multi-criteria optimization problems, the problems of stochastic and parametric programming.

The use of computers allows you to set the conditions of the problem, not only in the analitical form, in the view of formulas, but also with the help of tables, simulation programs, algorithms and solutions of differential equations and etc.

In this regard, the functions describing the optimized structure can have complicated non-linear nature, which does not allow to obtain an optimal solution in analytical form using the classical methods of differential and variational calculus. In this regard, in recent years began to develop intensively the methods of search optimization that provide a numerical solution of problems by means of computers. Unfortunately, even among search methods there is no universal method that would allow to obtain an optimal solution for all applications of nonlinear programming. Currently, the solution of each problem of optimal design, that are formulated as a nonlinear programming problem, you may need the several search methods, but even in this case, success will largely be determined by knowledge of the physical nature of the problem.

With the development of computer technology the opportunity, using a computer, create a probabilistic models of optimization of facilities, as well as to carry out their analysis, search and selection of based on search prehistory of search of models corresponding to a given optimality criterion. Simulation of different optimization objects, such as building structures, was the basis of the optimization technique, called random search based on the use of random and pseudo-random number sequences [1]. The random search methods are not without reason called the methods of statistical tests, keeping in mind that the basis for collecting information on the behavior of the objective function of random search methods is a random sorting of various possible states. This procedure is typical for a random statistical test methods.

As rightly noted by the authors [2], "... the solution of numerical problems by these methods (the methods of statistical tests) in spirit closer to the physical experiment than to the classical numerical methods." In fact, in the base of these methods "is lying the statistical modeling of the experiment using computer equipment and the registration of numerical characteristics obtained out of this experiment" [3].

(2)

International Journal of Emerging Technology and Advanced Engineering

162

For the first time the idea of the feasibility of random behavior clearly formulated W.R. Ashby in his monograph [9] and has implemented her in a well-known homeostat. The merit of W.R. Ashby was that he proposed to seek control, the stabilizing the system state not deliberately, but randomly. Further development of the ideas embodied in the homeostat, led to the use of random search as a management tool and to gather information and build on this basis of random search algorithms. Fundamental works of L.A.Rastrigin [1, 4, 10], L.A.Rastrigin, KK, Ripa, G.S.Tarasenko [5] are devoted to the theory of statistical research methods.

In these studies investigated the local properties of a variety of random search algorithms, mostly locally-adaptive, are evaluating the effectiveness of the considered algorithms. Is researched the integral behavior of stepper algorithms of random search, the work of search in conditions of interference, are consider the questions of self-study in the process of random search, coordinate-wise self-learning, when probabilistic properties are changing along each coordinate, and continuous, at which the predominant direction of the search can be any. Is considered also the issues concerning to the statistical methods of global search, are investigated a number of algorithms, is compared the convergence of different methods of statistical optimization, both at the theoretical and experimental way of learning and are researched the processes of learning in the systems of statistical search. To work of local stepper search algorithms without self-adaptation and storage (algorithms with paired trials with returning, with recalculation and with a linear extrapolation) in the absence of interference are devoted the earliest studies on the random search [11], where is determined the effectiveness of these algorithms, and is shown that the speed random search in the optimization of multivariable objects, ceteris paribus exceeds the speed of the gradient method. In [1] is studied the behavior of the same algorithms in the interference environment and the impact of non-locality random search in the presence of interference. The random search in the case of wandering objectives is considered in [1]. The questions of adaptation of step in random search algorithms discussed in [5, 12]. An algorithm with a continuous distribution of step magnitude allows to assign a priori initial step of any length and adjust it in accordance with the change of objective function. It is shown that the algorithm with a continuous distribution of step length is a generalization to the case sequential reduce of step. In [13] is proposed to smooth the objective function in a central field in the vicinity of the starting point and each intermediate from which it is made the next step, by the spherical hyper-paraboloid. For functions with ravines is proposed the approximating algorithm using the elliptical hyper-paraboloid [14].

The problem of global search is devoted the work [15]. The proposed algorithm of global search with controlled density distribution of trials within the permissible area ensures reliable operation at the search for global extremum not only for functions whose level lines of local extremums are sufficiently well approximated by, for example, by the hyper-sphere, but also for functions with ravines. In this case it is proposed to introduce rotating basis, allowing to orient the axis along the bottom of curvilinear ravines at each step. The adjust of the algorithm is performed on the test functions. In [11] discussed the problem of estimating the direction of the gradient descent by statistical method. To this end, is determined the density distribution of the angle between the gradient of the statistical function and the gradient of objective function to be optimized, is determined the average displacement along the gradient and dispersion, as well as losses on the search.

II. SETTING SEARCH ENGINE OPTIMIZATION TASKS

The objective of search engine optimization is a multi-step process of gathering information and making decisions on the basis of the information received. More acquainted with the execution of search engine optimization tasks the reader can in numerous literature, in particular in [16].

In real life we are accustomed to three-dimensional space. Mathematics operates with the n - dimensional space or hyper-space. The dimension of the space is determined by the number of independent control

variables

X



(

x

₁

,

x

₂

,...,

x

_n

)

, which are changing in the search process. Each of the control variables of vector Х defines one of the coordinate axes, the beginning of which is placed in a certain starting point of the parameter space, and which can be moved in the permissible area in the search process.

Vector of control variables is a vector argument of the so-called objective function or criteria of quality.

The area of permissible space of control parameters is allocated by a set of constraints, the physical meaning of which is determined by the statement of the problem.

The aim of the search is to reach an extreme state of the object and the determination of such a vector of

control variables

X

*





x

₁*

,

x

*₂

,....,

x

_n*



, that will deliver this state.

The problem of mathematical programming can be formulated as follows: find a vector of control variables



* *



2 * 1 *

,....,

,

x

_n

x



X

, (1)

That delivers the minimum (maximum) of the objective function:

)

,....,

,

(

)

(

F

x

₁

x

₂

x

_n

(3)

International Journal of Emerging Technology and Advanced Engineering

163

in the performance of restrictions:



(

),

(

),....,

(

)



)

(

X

₁

X

₂

X

G



g

_m , (3)

where the restrictions

g

_i

 

X

can take the form of:

 

X



0

i

g

;

g

_i

 

X



0

or

g

_i

 

X



0

.

III. THE MODELS OF OPTIMIZED OBJECT

The effectiveness of the search procedure is determined by several parameters, among which the most important are the speed and convergence. Usually the search effectiveness is determined by comparing different algorithms working in the same conditions.

In determining the same conditions is meant the form of the objective function and constraints, the area of search, the choice of a starting point, etc. In other words, for efficiency comparison of different search algorithms need to set such conditions, in which these algorithms are working. Adequate working conditions of the search system are largely dependent on the shape of the objective function of optimized object. Consider some of the most common features of models of objective function [1].

1. The linear model of the object. It is characterized by a linear dependence on the control parameters in the study area of the quality index of the object

   

 



0 0



2 0 1

0



x

,

x

,....

x

n

X

:





_



 











n

i

i i i

n

F

a

x

F

1

0 0

2

1

,

,...,

(4)

or in vector form:







grad

(

)



)

(

X

F

₀

X

₀

F

X

₀

F









(5

)

Within defined displacement



X



X



X

₀, the objective function is linear, and the gradient vector of the objective

function

grad

F

(

X

₀

)



(

a

₁

,

a

₂

...,

a

_n

)

assumed to be

constant in the search area. Where in

F

₀



F

(

X

₀

)

.

For sufficiently small steps



X

or far from the extremum, this model reflects well the local properties of the optimized object. It is enough to "smooth" the objective functions are well described by this model. For "not smooth" objective functions the linear model is used as a locally-linear model, i.e. linear model only in the area of one step of search.

The degree of information complexity of the model is determined by the probability with which a step in a random direction is successful, i.e.

0 



F

. The smaller the probability, the more information is required to make a successful step, i.e. to reduce the objective function, and the more difficult will the situation for the search.

2. The central model of the objective function is used for the analysis of search algorithms in the area of target:









































2 / 1

1

2 * 2

1

,

,...,

n

i

i i

n

f

x

F

, (6)

where f

function depending on the distance r to the target



* *



2 * 1 *

,....,

,

x

_n

x



X

in the parameter space:





1/2 1

*





















n

i

x

r

(7)

At large distances to the target the central model fits well with linear and at

r





_{coincides with it.}

For the central model, the probability of a successful step depends on the ratio of the distance to the target and stride length; with decreasing distance to the target, this probability decreases. Consequently, at the approaching to target the loss on search and retrieval difficulties increase.

3. The elliptical object optimization model, describes the

behavior of the object in the area of target

X

* as follows:









1/2

1

2 * *

2

1, ,...,

    

  

 





 n

i

i i i

n F a x x

x x x

(4)

International Journal of Emerging Technology and Advanced Engineering

164

where the coefficients

a

_i



0

are proportional to the components of the gradient of the objective function. At

n

a

₁



₂



...



this model degenerates into the central model (6). If one or more coefficients selected significantly smaller than others, such model is a ravine, directed along the respective axes, and its dimension is equal to the number of significant variables. The

significant variables are defined as variables

x

_i for

which the absolute value i

x

F



is substantially less than

for others. In this case, the significant parameters

x

_i

correspond to smaller values a_i.

IV. THE CHARACTERISTICS OF SEARCH EFFECTIVENESS

To evaluate the effectiveness of the search process are used by some of the characteristics derived based on selected criteria. [1]. The search characteristics are used for the purposes of synthesis, comparison and selection of algorithms, the most effective in current situation.

Among the existing search characteristics distinguish two classes: class local and nonlocal, integral characteristics.

The class of the local characteristics defines the effectiveness of search at one stage, i.e. on a series of search steps with a single value of working step. The second class includes the characteristics that determine the efficiency of the search process from the beginning to the end of the optimization.

One of the essential characteristic of the local search is the losses on a search. This local search feature determines the speed of search at one stage. Numerically losses search can be represented as a ratio of the expenditure of trials on stage of search to the resulting effect, where for the stepper search the expenditure is determined by the number of trials, and the resulting effect − by the change of the quality index:

)

(

)

(

i i i

F

M

N

M

K





, (9)

where М – a sign of the mathematical expectation;

i

N

 the number of trials (experiments) on i m search



F

_i the relative change in the quality index at the i m search phase, for example:

i i i

F

a

F

grad







, (10)

where а – the value of the working step.

Another important characteristic of the local search process is the probability of error. This characteristic determines the probability of occurrence in the search for erroneous working step.

Operating step considered erroneous if the quality indicator value in new state exceeds the quality indicator value in the initial state (at the necessity of minimizing of quality indicator value), and vice versa. Thus, the probability of error is:



X



X





X





p

F

(

)

F

(

P

, (11) where Х – vector of the initial state of the system;

X



 vector of working step.

Among nonlocal (integral) characteristics of search are the characteristics that determine its effectiveness from the perspective of certain criteria. Such criteria should be primarily the accuracy of the solution and the number of steps needed to solve the problems with the required accuracy.

By solving the optimization problem the inaccuracy on the steps of the search is the value of

*

)

(

F

_N







X



, (12)

N

X

 the solution of the problem, obtained on the basis N



F  the value of the quality function for the exact solution. It is assumed that the current solution Х* is only, i.e. it corresponds to one point in the space of optimized parameters

 

X

.

The value





_{is random. The average value of} inaccuracy of solving the problem of optimization on the basis of N trials is given by:













0

)

(

)

(



p



d



M

(13)

)

(





p

 the density of the distribution of

inaccuracy (at

p

 

0 

0

).

The number of samples N depends strongly on the initial conditions

X

₀. Therefore, for the convenience of comparison of algorithms we choose some specific initial

conditions, or calculate the average

N

(

X

₀

)

among all occurring initial situations





N

(

X

₀

)

p

(

X

₀

)

d

X

₀

N

_ , (14)

)

(

X

₀

(5)

International Journal of Emerging Technology and Advanced Engineering

165

The best algorithm is the one which provides a solution to the problem with the minimum number of search steps N and the of average accuracy of at least not

lesser a predetermined, i.e.







*.

V. THE LEARNING AND SELF-LEARNING SEARCH ENGINE

When applied to the optimization of multivariable systems, the direct method of trial and error, that is blind random search is inefficient. The disadvantage of blind scan is a low probability of finding a purpose.

Overcome this disadvantage one can, if the pre-investigate the optimization object and use additional information about him. Such additional information is contained in the value of the objective function at the optimization of object.

We introduce a measure of closeness in the set of

controls

 

X

, where

X





x

₁

,

x

₂

,...,

x

_n



[1]. Such

measure of the closeness between the controls

X

₁ and

2

X

is the distance between points

X

₁ и

X

₂ in n -

 

X

:

1 2 2 ,

1



X



X

r

(15)

This distance is usually associated with a change in the

objective function in the transition out of

X

₁ in

X

₂, i.e.

F

   

X

₁



F

X

₂







r

₁_,₂ , (16) where



_{is a constant value and the inequality exists}

in some domain space management

 

X

. Consequently, the smaller the distance between the controls, the less changes the objective function.

For most of real objects of optimization for

sufficiently close

X

₁ and

X

₂ the gradient values of the objective function at these points are also close, i.e. the value:

 





 

1

 

2

2 1

grad

X

F







(17)

is close to unity not only for sufficiently small values

r

₁_,₂. This makes it possible to use the estimate:

 

2

grad

 

1

grad

X



X

, (18)

It makes it possible to stipulate the behavior of quality

function at the point

X

₂ on the observations made at the

point

X

₁.

The search process is divided into several stages. At each stage is determined the own management objective.

 

i

F

X

is the state on the i-th stage. Then the search target at this stage is to find such a vector

X

_i_₁ for which



i



F

 

i

F

X

_₁



X

, (19) i.e. increase



F

should be negative:



₁

  





0 



F

X

_i_

F

X

_i (20)

The new management is sought in the area

X

_i by

introducing a randomness to change the state, which was reached on the previous search step

1

1 





i





i

X

, (21)

The positive reaction

R

_i on i-th stage in the process of such search appears when is reached the sub-target:



  













F

_i

F

X

_i ₁

F

X

_i

0 R

_i (22)

The negative reaction is associated with the opposite

result:



F

_i



0 

R

_i.

The "wandering" algorithm of random search takes the form:













_ _

R

X

R

Ξ

X

at

i

i 1 (23)

Random trials

Ξ

in (23) are made small in absolute value, in order the probability of achieving sub-target was big enough.

The algorithm [1] is built on the principle of "punishment" by randomness: the search system performs occasional steps in the space of control parameters, is yet would be found the step that would lead to a decrease in the objective function. Positive reaction of the algorithm is to repeat this step as long yet the quality score will not start to increase, which causes a - a random selection of new directions, etc.

The effectiveness of this search algorithm is guaranteed by the condition (18), which states that the

success that was achieved at the point

X

₁, could happen

again at the point

X

₂ if the distance between the points

1

(6)

International Journal of Emerging Technology and Advanced Engineering

166

Such form of behavior is certainly reasonable and expedient and is suitable for linear objects, and close to them, the properties of which with the transition from one state to another are changed slightly, i.e. the inequality (18) is performed. If this condition is not satisfied, the considered algorithm is of limited use.

For objects for which the inequality (16) and does not hold the inequality (18), it is more expedient to use an algorithm with the "encouragement" by randomness. In this algorithm, a negative reaction to the failure of R consists in the regular "correction" of search, i.e. each time the operations are performed the operations in order to eliminate the consequences of failure.

This algorithm can be written in the form [1]:













_ _

R

X

R

Ξ

X

at

F

at

i

i 1 (24)

As can be seen, the operator of the random step

Ξ

is entered as a positive reaction to a successful move that led to a decrease in the objective function (



F



0

).

The negative reaction

R

 causes the action (for example

F





X

_i









X

_i), aimed at overcoming the

resulting negative effects (



F



0

), and then the random step

Ξ

_{is performed again. Thus, operability of} this search algorithm is obtained by correcting errors that were made in the process of random samples.

When constructing algorithms (23) and (24) it was provided that the statistical properties of the random operator steps

Ξ

_{remain the constant in the search} process. Using the prehistory of search and the purposeful reconstruction of the statistical properties of the operator

Ξ

, we can significantly improve search performance. Built on this principle the random search algorithms acquire the ability for learning. This self-learning contributes to the search for additional feedback, which is a reaction of higher order in the behavior of the object in the search process than the direct reaction R.

Algorithms that use the self-learning, would adduced in next papers, which examines issues of adaptation of the distribution of the direction of the search step.

REFERENCES

[1] −

376 p.

[2] Buslenko I.P., Golenko D.I., Sable I.U., Sragovich V.G., Y.A.

−

[3] Buslenko I.P., Schrajder Yu. A. The method of statistical tests. − p.

[4] Rastrigin L.A. A random search in optimization problems multivariable systems. - Riga: Zinatne, 1965. − 287p.

[5] Rastrigin L.A., Ripa K.K., Tarasenko G.S. Adaptation of random search. Riga: Zinatne, 1978. − 243 p.

[6] Gurin LS., Dymarsky Yu.S., Merkulov P.X. The problems and methods of optimal allocation of resources.. − M .: Sov. Radio, 1986. − 513 p.

[7] Bocharov I.N., Fel'dbaum A.A. Automatic optimizer for finding the minimum out of a few lows // Automation and

−- № 3. − T.23. − p.67−73.

[8] Каrnopp D.C. Random search techniques for optimizations problems // Automatica.  Vol. 1.  Pergamon Press. 1965. p.116123.

[9] Ashby W.R. The structure of the brain. - M .: IL, 1962. − 367p. [10] Rastrigin L.A. Random search with linear tactics. − Riga: Zinatne,

1971. − 310 p.

[11] Volynskij E.I., Malyshev S.A., Filatov G.V. About of estimation of the descent directions with use of statistical gradient method // Automation and Computer Science. -

−55.

[12] Volynskij E.I., Filatov G.V. On the question of adaptation step in the algorithms of random search // Automation and Computer

−

[13] Volynskij E.I., Poteshkina D.I. , Filatov G.V. The application some of smoothing operators in the problems of search of extremum // Math. AH the USSR. Technical cybernetics. −

−

[14] Volynskij E.I., Filatov G.V., Khablo T.V. About one gullied global search algorithm // Automation and Computer Science. −

−54.

[15] Volynskij E.I., Filatov G.V., Khablo T.V. Some algorithms of global statistical optimization // In Proc. "Numerical methods of nonlinear programming." −

p.131−