In Section4.3, a previously introduced exploitation method was discussed. A key complexity issue for high-dimensional problems was identified and a new approach to construct the neighbourhoods was introduced. Section4.4introduced a fuzzy-based mechanism to assign weights to each neighbour, based on cohesion and adhesion. All these concepts are now brought together into a new approach that can take the place of the LOLA algorithm in LOLA-Voronoi.
4.5.1
Fuzzy local linear approximation
Algorithm 1 Fuzzy LOLA (FLOLA): this exploitation algorithm computes a score ∀x ∈ X, indicating the non-linearity of the region surrounding x. New samples are chosen in the neighbourhood of theNnewhighest ranked samples.
Require: X, F , ζc,ζal,ζah,Nnew
initiateS (Section4.4.2) Calculate distance matrix for x for all xr∈ X do
Computeρ(xr) (Equation (4.4))
InitialiseN (xr) (Equation (4.3))
DetermineC(xr) and A(xr) (Equations (4.5) and (4.6))
Compute weights ω by evaluatingS Estimate g (Equation (4.1)), given ω
Calculate error on gradient estimation (Equation (4.2)) end for
PickNnewsamples with highest non-linearity score
Xnew= new samples in the neighbourhood of these samples
X = X∪ Xnew
The weights computed byS can be used to solve Equation (4.1) as a Weighted Least Squares3problem to estimate the gradient. After obtaining g, we can compute the non-linearity score. An overview of this new approach, known as Fuzzy Local Linear Approximation (FLOLA) is given in Algorithm1. Because the size of the system Equation (4.1) is not dependent on the size of the setX, only the for loop contributes to the complexity of the algorithm: this results in a complexity ofO N which is a massive improvement compared to LOLA. Furthermore, the for-loop allows parallel computation since each iteration is independent. The biggest cost are many distance calculations to determineρ(xr), A(xr) and C(xr). This is solved
by computing a distance matrix once prior to the for-loop: this matrix contains all
3Note that the weights are computed for each point x separately. This essentially means we turned
required information for computations inside the loop. Furthermore, for the next iteration of the sequential design process this distance matrix may be expanded by adding rows and columns avoiding a quadratic complexity. Distances matrices tend to occupy a lot of memory in case of many points, which is unlikely for this algorithm in the context of surrogate modelling as each evaluation is expensive. Due to the limited size of the setX, the size of the matrix remains manageable.
4.5.2
Including an exploration metric
Similar to LOLA, the exploitation based algorithm Fuzzy Local Linear Approxima- tion (FLOLA) can be complemented with a Voronoi approximation based explo- ration component and form FLOLA-Voronoi. For each point xr, the non-linearity
scoreEfuzzyis complemented with a measureV indicating an approximation of
the relative Voronoi cell size of the reference point. For more information on approximating the size of a Voronoi cell, the reader is referred to [5]. The value of V is in the range [0, 1] so Efuzzyis first normalised and then added toV :
Hfuzzy(xr) = V (xr) +
Efuzzy(xr)
PN
i=1Efuzzy(xi)
. (4.7)
For clarity, the pseudocode of FLOLA-Voronoi is shown in Algorithm2. The only difference with LOLA-Voronoi is the algorithm used to calculateEfuzzy(xr). The
hybrid scoreH is then used to rank all currently available points according to the non-linearity and the sample density of the surrounding region. TheNnewhighest
ranked reference points are selected to assign new points in the next iteration. The position of the point is determined by considering local space-fillingness. Usually the position maximising the minimum distance from both the reference point as well as its neighbours is chosen.
The combination of both criteria guarantees we do not get stuck in one region of the design space and no large areas are left unexplored. However, the exploitation score pushes the strategy to sample non-linear regions much denser when they are discovered. When these regions are sampled dense enough, the FLOLA score will lower, and exploration will take over. This additional information on non-linear regions helps the surrogate model to capture the non-linear behaviour accurately as more information is provided on irregularities. In Equation (4.7), the exploration and exploitation component contribute equally. It is possible to use a different balance, or even change the balance dynamically as more samples become available. For more information, the reader is referred to [22].
SettingNnew= 1 is optimal, as each sampling decision can be made with the latest
it is exploited immediately. However, adding add more samples each iteration does not lead to undesired clusters or a bad design, since only a single additional point can be placed in each Voronoi cell during one iteration. For high-dimensional problems this is recommended as fitting a surrogate model may be expensive.
Algorithm 2 FLOLA-Voronoi: hybrid sequential strategy. Combines an exploita- tion and an exploration score (FLOLA and Voronoi respectively) and selects a new candidate samples in the neighbourhood of theNnewhighest ranked samples.
Require: X, F , ζc,ζal,ζah,Nnew
for all xr∈ X do
CalculateEfuzzy(xr) (Equation (4.2))
CalculateV (xr) (See [5])
ComputeHfuzzy(xr) (Equation (4.7))
end for
SortX by Hfuzzy
fori = 1 to Nnewdo
xnew← location near xi
Pnew ← Xnew∪ xnew
end for