Surrogate Assisted Evolutionary Algorithm for Multi-Objective
Algorithm 5.1 Surrogate Assisted Evolutionary Algorithm Require: N G > 1 {Number of Generations}
Require: M > 0{Population size}
Require: IT RAIN > 0{Periodic Surrogate Training Interval}
1: A = ∅ {Archive of Actual Evaluations}
2: P1 = Initialize() 3: Evaluate(P1) 4: A = AddToArchive(A, P1) 5: S = BuildSurrogate(A) 6: for i = 2 to NG do 7: Rank(Pi−1) 8: repeat 9: p1, p2 = Select(Pi−1) 10: c1, c2 = Crossover(p1, p2) 11: Mutate(c1) 12: Mutate(c2) 13: Add c1, c2 to Ci−1
14: until Ci−1 has M children
15: if modulo(i− 1, IT RAIN) == 0 then
16: A = AddToArchive(A, Pi)
17: S = BuildSurrogate(A) 18: end if
19: Evaluate(Ci−1,S)
20: Pi = Reduce(Pi−1 + Ci−1)
21: end for
22: Evaluate(PNG){Final population re-evaluated}
archive. These solutions are identified by the k-Means clustering algorithm. Such an approach is in accordance with the suggestions by Haykin (1999) and Jin (2005) that using the entire archive could lead to over-fitting and the introduction of false optima.
5.2.1 Initialization
All the solutions in a population are initialized by selecting individual vari- able values as given in Eq. (5.2).
136 T. Ray, A. Isaacs and W. Smith
where xi denotes the initialized variable, andU[0, 1] is an uniform random
number lying between 0 and 1.
5.2.2 Actual Solution Archive
All the unique candidate solutions are maintained in an external archive. These solutions are used to train the surrogate models periodically after every S generations. Each entry in the archive comprises the candidate
design (x) normalized in the range [0, 1], objectives (f1, . . . , fm) and con-
straints (g1, . . . , gp) evaluated at x. To avoid numerical difficulties in fitting
surrogate models, a new candidate solution is added to the archive only if none of the solutions in the archive lie in the close neighborhood of that so- lution (computed using the Euclidean distance). The neighborhood radius
of 1.e− 6 is used for this study.
5.2.3 Selection
The procedure for selection of parents is the same as that of NSGA-II. Binary tournament is used to select a parent from two individuals. Binary
tournament between two candidate solutions x1 and x2 is performed as
follows:
(1) If x1is feasible and x2 is infeasible: x1 is selected and vice versa.
(2) If both x1 and x2 are infeasible: the one for which the value of the
maximum violated constraint is smaller is selected.
(3) If both x1and x2are feasible and x1dominates x2: x1 is selected and
vice versa.
(4) If both x1 and x2 are feasible and neither dominate the other: one of
x1and x2 is selected randomly.
Four solutions are chosen at random from the population. From the first two solutions, parent 1 is selected and from the last two individuals, parent 2 is selected using binary tournament. To ensure that all the solutions in the population take part in the selection process, a shuffled list of IDs (1 to M ) is created and individuals are picked up 4 at a time from this list.
5.2.4 Crossover and Mutation
Simulated Binary Crossover (SBX) [Deb and Agrawal (1995)] operator for real valued variables is used to create two children from a pair of parents.
August 25, 2008 19:30 World Scientific Book - 9in x 6in book
Surrogate Assisted Evolutionary Algorithm for MOO 137
The offsprings y1and y2are created from parents x1 and x2by operating
on one variable at a time as shown in Eq. (5.3),
yi1= 0.5 [(1 + βqi) xi1+ (1− βqi) x2i]
yi2= 0.5 [(1− βqi) x1i + (1 + βqi) x2i]
(5.3)
where βqi is calculated as,
βqi= ⎧ ⎨ ⎩ (2ui)1/ηc+1, if ui≤ 0.5, 1 2(1−ui) 1/ηc+1 if ui> 0.5. (5.4)
and where ui is the uniform random number in the range [0, 1) and ηc is
the user defined parameter Distribution Index for Crossover.
The Mutation operator used is the polynomial mutation operator de- fined by Deb and Goyal (1996). Each variable of y is obtained from a corresponding variable of x as given in Eq. (5.5).
yi= xi+ (xi− xi) ¯δi (5.5)
where ¯δi is calculated as,
¯ δi= (2ri)1/(ηm+1)− 1, if ri< 0.5, 1− [2(1 − ri)]1/(ηm+1), if ri≥ 0.5. (5.6)
and where ri is the uniform random number in the range [0, 1) and ηm is
the user defined parameter Distribution Index for Mutation.
If the value of any of the variables calculated using crossover and mu- tation operators falls below the lower bound (or above the upper bound), the value of that variable is fixed at the lower bound (or the upper bound).
5.2.5 Ranking
Ranking of the candidate solutions involves ranking the feasible and the infeasible solutions separately. The feasible solutions are first sorted using non-dominated sorting to generate fronts of non-dominated solutions. So- lutions within each front are ranked based on the decreasing value of the crowding distance. The infeasible solutions are ranked based on increasing order of the maximum violated constraint value.
5.2.6 Reduction
The reduction procedure for retaining M solutions for the next generation from a set of 2M solutions (parent and offspring population) uses the ranks obtained by the ranking procedure.
138 T. Ray, A. Isaacs and W. Smith
(1) If the feasible solutions number more than M ,
• the M solutions are selected in the order of non-dominated fronts
and decreasing crowding distance in each front. (2) If the feasible solutions are less than or equal to M ,
• all the feasible solutions are selected in the order of non-dominated
fronts and decreasing crowding distance in each front, and
• all the remaining solutions are selected from the infeasible solu-
tions in the order of increasing value of the maximum constraint violation.
5.2.7 Building Surrogates
Outlined in Algorithm 5.2 are the steps involved in building surrogates for all the objectives and the constraints evaluated using the actual analysis of the solutions in the archive. A fraction (α) of solutions in the archive is used to build the surrogates to prevent over-fitting. These solutions are identified
by the k-Means clustering algorithm where design variables x1, . . . , xn are
used as the clustering attributes. To restrict the time required to train the RBF model, an upper bound is put on the maximum number of solutions
(Kmax) used to train the surrogate models. A subset (As) of the archive is
created by selecting the solutions closest to the centroids of the K clusters obtained by the k-Means algorithm. The subset of the archive is then used to build surrogates for each of the objectives and the constraints.
5.2.7.1 Radial Basis Functions
Radial Basis functions (RBF) belong to the class of Artificial Neural Net- works (ANNs) and are a popular choice for approximating nonlinear func- tions. RBF φ has its output symmetric around an associated centre µ.
φ(x) = φ(x − µ) (5.7)
where the argument of φ is a vector norm. A common RBF is the Gaussian function with the Euclidean norm.
φ(r) = e−r2/σ2 (5.8) where σ is the scale or width parameter.
A set of RBFs can serve as a basis for representing a wide class of functions that are expressible as linear combinations of the chosen RBFs
August 25, 2008 19:30 World Scientific Book - 9in x 6in book
Surrogate Assisted Evolutionary Algorithm for MOO 139
Algorithm 5.2 Building of Surrogates