A description of a typical optimization process has been presented in this chapter. In- variably, this process is automated. The description of the automatic progression of procedures used in the work covered in this thesis is outlined.
Following the general structure of an optimization process, a parametric geometry must be constructed. A number of techniques which provide the building blocks for curve parameterization have been discussed. A DoE approach is then used to provide a set of geometries which fill the design space efficiently. Analysis is performed on these geometries providing the responses through which a response surface can be built. An update point is found using the RSM and its objective function is found. If the time to build the RSM is still reasonable, then the RSM is rebuilt including the recently analysed update point and searched for the next update point. Once the time taken to build the RSM becomes prohibitively expensive, the design may not have found an adequate objective function, especially for design problems of high-dimensionality. If the desired objective function has been satisfied then the optimum has been found. If not, then an exploration of a reduced area of the design space can be performed around this point to converge quickly to an optimal design. This description of a typical optimization strategy has not elaborated upon the details of response surface modelling, in particular Kriging, and how the most appropriate update point is located. Details of this can be found in the following chapter.
Optimization using Response
Surface Methodology
3.1
A Global Approximation
For the majority of engineering problems, the shape of the response surface is unknown. Many objective functions are deceptive and can lead to an optimization process based on a local approximation, which will only find a local maximum or minimum. In light of this, it is paramount that an approximation based optimization routine progresses towards the global optimum, or at least explores the design space reasonably widely. RSMs can be one of two varieties: an interpolating model or a regression model. The difference is that the construction of the surface occurs through the set of points that have been sampled, in the case of interpolation, or near, in the case of regression. For functions whose objective function curve or surface is not known a priori, the decision as to whether the response surface should be allowed to feature regression is not ob- vious. For a completely regressing response surface, the addition of more points may not necessarily lead to a more accurate representation of the function, whereas the ad- dition of more points in an interpolating surface involves more points of full accuracy. However, it is important to note that unless the function is smooth and continuous, the interpolating surface approximation may not represent the function very accurately
in between the sampled points, since the requirement for interpolation can lead to the surface ’overshooting’ or ‘undershooting’ the objective function values in such regions. The definition of an RSM can differ widely. Montgomery (2000), for example, uses the term for polynomials, while this thesis uses a more general definition. Here, two sorts are considered: polynomial models and radial basis function (RBF) models.
Suppose a k-dimensional problem is sampled at n points X ={x1,x2, . . . ,xn}, where each xi (i = 1, . . . , n) is a vector containing k design variables xi1, xi2, . . . , xik. Each sample point has a corresponding objective function value yi, collectively giving the vector y={y1, y2, . . . , yn}.
Polynomial models can be generalized by
ˆ y(x∗) = n X l=1 alαl, (3.1)
wherex∗ is the untried point,a
lare coefficients and αl is a basis function where a set of all polynomials inxwith degreedcan be generated by a basisB={αl(x)|l= 1, . . . , n}. For example, ifd= 1, a basis will contain a combination of the constant 1 and any first order term of x. Ifd= 2, a basis will contain a combination of the constant 1 and any second order term ofx and so on.
These are the simplest of global RSMs but as the objective function landscape becomes more complex, higher order polynomials are required which increases the number of points needed to build the response surface. Hence, low order polynomials are generally considered for this type of approximation and because of this, there is a concern that it may lead to large inaccuracies in the model. Radial basis functions differ from polynomial models only in their choice of basis function. RBFs can be generalized by
ˆ y(x∗) = n X t=1 atΦ (x∗−xt), (3.2)
where the new untried pointx∗ is related to all of the sample points, atare coefficients, and Φ is commonly known as the kernel. There are a variety of different forms that Φ
can take to produce different basis function RSMs. A number of different examples can be defined as follows.
Φ (x∗−xt) = kx∗−xtk linear spline
= kx∗−xtk2log (kx∗−xtk) thin-plate spline
= kx∗−xtk3 cubic spline = exp−kx∗−xtk σ2 Gaussian (3.3)
All of the models mentioned above can be solved in such a way to form interpolating or regression models depending on the number of bases (Keane, 2004). Fourier analysis methods and least-squares methods are just two of the many techniques available to solve the models mentioned above.
Kriging (Matheron, 1963; Cressie, 1990) is a technique in the RBF category and is generalized as Φ = exp − k X s=1 θs|x∗s−xts|ps ! , (3.4)
where θs and ps are unknown coefficients, commonly known as the hyperparameters. These hyperparameters provide statistical information on the quality of the surface being built and, once tuned, they can be used to rank the design variables in accordance to their relative dominance, see Jones (2001) and Keane (2003). The hyperparameters and their tuning will be discussed further in section 3.2. In most practical applications, the above Gaussian function is used and is based on that given by Sacks et al. (1989). Low order polynomial approximations have been shown to provide a poor global approx- imation to some problems. However, Jones (2001) discusses the relative merits of the Kriging technique’s robust capability in finding the true global optimum given poten- tially deceptive functions. Due to the versatility of Kriging in its capability of approxi- mating complex objective function landscapes and its provision of additional statistical information of the surface, all cases studied here use a Kriging RSM for optimization purposes.
3.1.1 Kriging
Kriging is a technique which provides a statistical interpretation so that, in addition to the interpolator (or ‘predictor’), a measure of the possible errors in the model is ascer- tained, which in turn may be used to position any further design points more prudently. The response surface can represent an approximation of the objective function, the er- ror of the approximated objective function values, or the expected improvement in the objective function value that can hypothetically be attained over the design space. A balance between global exploration of the design space and local exploitation of promis- ing regions of the design space is sought. Searching over the approximated objective function, the error of the approximation or the expected improvement of the approx- imation provide different balances between exploration and exploitation. These three methods are discussed in further detail in section 3.2.1 in order to determine the best method for finding a global optimum design.
Although Kriging is a versatile and robust method of approximating an objective func- tion, it should be noted, however, that Kriging is not suited to all practical applications. Its efficiency is dependent largely upon the number of design variables defining the prob- lem. In section 2.2 it has already been mentioned that strong shape control is important and in many cases, this is achieved by increasing the number of design variables. There- fore, one has to decide whether to trade-off the geometric complexity of the model for the computational time required to obtain an adequate solution. Typically, Kriging is computationally practicable for up to approximately 20 design variables (Keane, 2003).