Framework of surrogate-based optimization

The basic strategy of using surrogate models in optimization is quite intuitive. The first

step concerns the generation of an initial sample of vectors which are evaluated with the

true expansive function and used to train a metamodel. Subsequently, the main optimiza-

tion search is invoked. The generation of the sample vector deeply impact the overall

search effectiveness as well as the metamodel accuracy and its predictive capability inside

the design space.

A plethora of strategies have been proposed not only to validate the surrogate, but also

to enhance its accuracy by adding a feedback loop in which the surrogate optimum design

must be confirmed with calls to the true function and used to update the sampling. In so

doing, it is possible to categorize the surrogate-based optimization into two groups:

I Simple-level framework: the optimization is entirely driven by the surrogate model

II Bi-level framework: the true function is used to evaluate the surrogate optimum

designs

These approaches lead to different problems and limitations and are briefly analyzed in

this section.

Simple-level framework

In this context all the solutions have been assessed in the SM and the achieved fitness is

assumed to be comparable to that assessed by the real function. Since prediction with

the SM is much more efficient than that by the expensive analysis code, the optimization

efficiency can be greatly improved. In addition, SMs also serve as an interface between

the analysis code and the optimizer, which make the establishment of an optimization pro-

cedure much easier. The comparison of the conventional and the surrogate-based simple-

level optimization is sketched in Figure 3.6.

(a) (b)

FIGURE

3.6: Comparison of conventional (a) and surrogate-based (b)

simple-level optimization probelm.

This approach is commonly seen in literature ( [88], [89] and [90] just to mention a

few), because the use of a such a simple approach seems to be the most straightforward

in using SM. However, it should be used carefully since its behaviour is highly dependent

on the accuracy of the SM. In fact, when a SM is not properly selected, or it is constructed

with a reduced-size training sample, or the sample is unevenly distributed, the constructed

model will usually be inaccurate. Therefore, if the optimization calculates the entire set

of solutions exclusively with the SM, the entire approach will have more probabilities of

converging to a false optimum, that, in a MOOP, is a Pareto front not corresponding to the

true Pareto front in the real function.

Bi-level framework

It is evident the need to call the true functions (the expensive analysis codes) not only to

validate the optimum solutions of the SM, but also to enhance the accuracy of the SM it-

self, by adding new sample points to the current sampled data set. To this scope, the entire

process can be regarded as a bi-level optimization problem, as sketched in Figure 3.7; the

main optimization concerns the creation and refining of the SMs and needs calls of the

true functions. The sub-optimization uses the current SMs to determine the new sample

sites by using any optimization algorithm such as gradient-based or EA.

F

IGURE

3.7: Flowchart of a bi-level surrogate-based optimization.

The selection of new points at which to call the true function, so-called infill points,

represents the heart of the surrogate-based optimization process. Applying a series of infill

points, based on some infill criteria, is also known as adaptive sampling (or updating),

that is we are sampling the objective function in promising areas based on a constantly

changing surrogate. It is therefore important to distinguish between the initial sampling

step which is employed prior to the main optimization search, and the infill sampling step

3.4. Surrogate-based optimization

51

which is performed during the search. Since the infill sampling vectors are iteratively

generated based on an optimization search, the objective functions affect the sampling.

The success or failure of a surrogate-based optimization rests on the correct choice

of model and infill criteria, or in other terms, in the balance between exploration and

exploitation.

Elements of exploration in the infill criterion need the research of global optimum

location. However, pure design space exploration can essentially be viewed as filling

in the gaps between existing sample points. Pure exploration is of dubious merit in an

optimization context [86] because time spent accurately modelling suboptimal regions is

time wasted when all we require is the global optimum itself. Exploration based infill has

its niche in design space visualization and comprehension where the object is to build an

accurate approximation of the entire design landscape to help the designer visualize and

understand the design environment they are working in or when the final SM is to be used

in a realtime control system.

On the contrary, exploitation-based infill criteria are attractive methods for local op-

timization. However, exploiting the surrogate before the design space has been explored

sufficiently, may lead to the global optimum lying undiscovered.

Thus, great caution must be taken in the choice of the infill criteria and a lot different

methods have been proposed; Jones [91] proposed a classification of the infill criteria into

two breeds:

• One-stage approach: the SM is not fixed when calculating the infill criterion,

rather the infill criterion is used to calculate the SM. Goal Seeking and Conditional

Lower Bound are often used

• Two-stage approach: the SM is fitted to the data and the infill criterion calculated

based upon this model: Searching Surrogate Model (SSM), Expected Improvement

and Statistical Lower Bound are the more common

The two-stage approach is far apart the most used. In particular, SSM is the more attractive

because its simplicity and applicability for all the SMs; in this method, an optimizer such

as EA is invoked to find the optimum, which in turn can be employed to refine the SMs.

Such an approach has been found to be very efficient for local exploitation in the design

space. This infill criterion highlights, once again, the importance that EAs have in MOOP

and their capability to fulfil surrogated based approaches.

In document Metamodel-based design optimization in industrial turbomachinery (Page 61-63)