Design of an efficient mutation operator

In [26], a runtime analysis of (1+1)-ES with adaptive isotropic Gaussian mutation on the sphere function is reported showing that (1+1)-ES is efficient on the sphere. This analysis applies unchanged to the Euclidean cone landscape.

In the previous section we claimed that (SGMR) is a semantic ε-mutation. That means that the Euclidean distance between the output vector of the parent and the output vector of the offspring is bounded by some fixed constant ε. That does not say anything on the way the output vector of the offspring is mutated, and particular it does not guarantee that the change in the output vector is isotropic Gaussian.

So, in this section we will designing a semantic mutation operator for real functions that on the output vector space always corresponds to an isotropic Gaussian mutation. We will then be able to reduce the runtime analysis of GSGP for any basis functions regression problem, i.e., any choice of basis functions gj, any choice of function to approximate p,

and any choice of training set T , to the above settings, getting the same guarantees on the runtime provided by the analysis of (1+1)-ES with adaptive isotropic Gaussian mutation. The semantic mutation we consider is of the form o(x) = p(x) + ms · r(x) where p(x) ∈ H is the parent function, ms ∈ R+ _{is the mutation step, r(x) is the perturbing}

function which is sampled according to some probability distribution over the functions search space H. As o(x) is a linear combination of elements of H it is also in H. For a fixed number of basis functions m, we can represent functions in H by their (fixed- length) vectors of coefficients in the linear combination. The semantic mutation on this representation becomes co = cp + ms · cr where co, cp, cr ∈ Rm denote the vectors of

coefficients of o(x), p(x) and r(x). Simplification of the offspring in this representation is implicitly achieved by algebraic operations on real vectors. The genotype-phenotype mapping on this representation becomes a function P : Rm _{→ R}k _{that maps vectors of}

Lemma 1. The genotype-phenotype mapping P is a linear map. So, it holds that P (c1+

c2) = P (c1) + P (c2) and P (λc) = λP (c) for all c1, c2, c ∈ Rm and λ ∈ R.

Proof. As explained in Section 5.1, given:

• Function: F (X) = c1g1(x) + . . . + cmgm(x)

• Coefficient vector C = (c1, . . . cm)

• Output vector: O = (o1, . . . , ok) = (F (X1), · · · , F (Xk))

The mapping P from C to O can be expressed in matrix form as O0 = GC0

(o1, . . . , ok) =          g1(X1) g2(X1) · · · gm(X1) g1(X2) g2(X2) .. . ... . .. ... g1(Xk) g2(Xk) · · · gm(Xk)                   c1 c2 .. . cm         

From linear algebra, we know that this is a linear map.

Consequently, line segments and circles on the space of vectors of coefficients (geno- types) are projected via P to, respectively, line segments and (possibly rotated and re- scaled) ellipses on the output vectors space (phenotypes). To obtain a circle on phenotype space, one needs an ellipse in the genotype space that compensates for the transformation P . In terms of semantic operators, this means that a line crossover on genotypes induces a line crossover on phenotypes. A carefully chosen non-isotropic Gaussian mutation on the space of genotypes corresponds to an isotropic Gaussian mutation on the space of phenotypes. We prove the last statement formally.

Theorem 12. The non-isotropic Gaussian mutation on genotypes co = cp + ms · cr

with cr ∼ N (0, G+IG0+) induces the isotropic Gaussian mutation on phenotypes P (co) =

Proof. co = cp + ms · cr; P (co) = P (cp + ms · cr); For linearity of P : P (co) = P (cp) +

ms · P (cr); P (cr)0 = Gc0r is a linear transformation of the multivariate Gaussian cr:

P (cr) ∼ N (0, G · G+IG0+· G0) = N (0, I · I · I0) = N (0, I)

The implementation of the semantic mutation requires to sample crfrom N (0, G+IG0+).

This can be done by multiplying G+ _{by a vector formed by sampling k times the normal}

distribution N (0, 1).

Using this we can design an operator that, acting on the vector of coefficients of the parent function, produces an Isotropic Gaussian Mutation on its output vector. The operator (IGSM) is presented in the following:

Definition 14. Given a function F : Rn_{→ R in the class H of functions generated by the}

functions base (g1(x), . . . , gm(x)), and C = (c1, . . . , cm) the vector of coefficients defining

F in H (thus F (X) = c1g1(x)+. . .+cmgm(x)). The Isotropic Gaussian Semantic Mutation

(IGSM) returns as offspring the function F0 ∈ H defined by the vector of coefficients C0

such that C0 = C + Cr and Cr ∼ N (0, G+IG0+).

5.3

Runtime Analysis

The main problem in implementing the IGSM presented above is that, in order to sample Cr ∼ N (0, G+IG0+), the algorithm needs to know the matrix G, which depends on the

training inputs (but not on the training outputs).

We will start analysing GSGP on a semi-black-box setting (Section 5.3.1), in which the algorithm needs to know the training inputs to calculate the matrix G, but not the output corresponding to each input (so given T = {(i1, o1), ..., (ik, ok)}, the algorithm has

just access to (i1, . . . , ik)). In Section 5.3.2 we propose a method to calculate the matrix