As stated in the introduction the double aim of the thesis is to advance the theoretical knowledge of Parallel Evolutionary Algorithms and Genetic Programming and to use this novel knowledge to produce new theory grounded efficient designs. In this chapter we have presented a review on the state of the art of the theoretical foundation of Genetic
Programming and Parallel Evolutionary Algorithms. There are many gaps to fill.
Concerning the theoretical foundation of Genetic Programming, we have stated that there are two leading theories. The first, based on schema theory and Markov chains analyses proper Genetic Programming evolving functions (i.e. tree based GP using swap tree crossover) but fails in providing quantitative results on the runtime. The second, inspired to the runtime analysis of evolutionary algorithms, can provide runtime results but it fails in doing that for proper GP, since it takes into consideration problems in which the fitness of an individual does not depend on its computed function (i.e. its semantics) but rather on its structure (i.e. its syntax). Geometric Semantic Genetic Programming (GSGP) gives the opportunity to fix the inadequacies of both the leading theories, thus allowing us to produce analysis leading to runtime results for problems in which individuals are treated as functions rather than as structures. Still GSGP has not been analysed rigorously yet and it also leaves open the problem of designing efficient operators for different domains that are provably good. Our contribution in Chapters 3, 4 and 5 will provide rigorous analysis of GSGP and design of provably efficient operators for the domains of boolean functions learning, classification trees learning and basis function regression.
Concerning Parallel Evolutionary algorithms there is no rigorous analysis on how is- land models perform on NP-hard problems, or how they deal with multi-objective fitness functions. Moreover previous studies took into consideration just homogeneous island in which all islands run the same algorithm and they did not consider heterogeneous models where each island can run different algorithms with different parameters, different op- erators, and even different fitness functions. This gap will be filled in Chapter 6 where an homogeneous island model in which each island solve a multi-objective formulation of the NP-hard problem SetCover is analysed. The analysis will inspire the design of a more efficient heterogeneous island model which assigns to each island a different single-objective problem representing a subset of the whole search space.
Particularly the works available on the migration interval are mainly experimental and a rigorous approach analysing schemes to adapt it throughout the run is missing. This gap will be filled in Chapter 7, where two new adaptive schemes for the migration interval are proposed. We will show, through rigorous analysis, that these new schemes can perform as well as a scheme fixing the migration interval to its best value in terms of upper bounds for the parallel runtime, while sometime guaranteeing even better upper bounds for the communication effort.
CHAPTER 3
GEOMETRIC SEMANTIC GENETIC
PROGRAMMING FOR BOOLEAN FUNCTIONS
In the previous chapter we introduced geometric semantic genetic programming in a general way. In this chapter we will design geometric semantic operators for the domain of Boolean functions and we will analyse the runtime of GSGP using these operators solving the problem of black box Boolean functions learning.
This chapter will provide the first example in this thesis of theory-driven design. We will first introduce and analyse a point mutation operator (see Definition ??), which was firstly introduced in [59]. The analysis will show the inadequacies of this operator and will guide towards the design of new block operators, which we will prove to be more efficient. This chapter is based on [61, 49]. My contribution consisted in the design of all the novel mutation operators, in the statements and proofs of Theorems 2, 3, 4, 5 and in the design and implementation of the experiments in Section 3.5 concerning Cartesian GP.
3.1
Black box Boolean learning problem
Given a complete truth table C = {(x1, y1), ..., (xN, yN)} consisting in the complete de-
scription of the input-output behaviour of a fixed Boolean function h : {0, 1}n → {0, 1} in n variables (N = 2n). A training set T consisting in τ ≤ N test cases T ⊂ C = {(x1, y1), ..., (xτ, yτ)} is sampled from the truth table uniformly at random without re-
placement.
The aim is to use just the training set T to learn a Boolean function h : {0, 1}n → {0, 1}
matching as well as possible the input-output behaviour described by C.
In this and in the following Chapter our algorithms will try to do that finding an ex- pression matching the input-output behaviour described by training set, thus minimising the training error
εt(h) = 1 τ X (xi,yi)∈T I[h(xi) 6= yi]
where I[·] is the indicator function that is 1 if its inner expression is true and 0 otherwise. The problem has the additional constraint of being black-box. Which means that the learning algorithm has no direct access to T . It has instead access to an oracle that, given a candidate Boolean expression X will return how well it matches the input-output behaviour described by T .
When τ = N , the training set encompasses all the possible input-output cases. In this situation finding an expression fitting the training set T will also lead to an expression fitting the complete set, and thus the original Boolean function. When, on the other hand, the training set is smaller than the complete set, fitting the training set would lead to a generalization error which can be defined as
εg(h) = 1 N X (xi,yi)∈C I[h(xi) 6= yi]
In the following sections we will consider just the problem of minimizing the training error εt(h), thus we will considered the black box Boolean learning problem “solved” when
an expression matching completely the training set is found. We will give considerations on the generalization error in Section 4.5.