Disembodied: The fitness evaluation is performed outside the body and the EA has no direct access to the world through the eyes of the robot. This approach might also make use of variables that are available to an external observer.
Section2.6.2will discuss the interaction between embodiment and the evolutionary process in more detail.
Which form the evolutionary algorithm takes depends on the extrinsic constraints of the application. For example, in this thesis where the focus is on continuous autonomous adaptation in a swarm of robots, the algorithm needs to be distributed and online, and use embodied fitness evaluations.
2.4 Algorithms for Distributed Online Evolution
2.4.1 Probabilistic Gene Transfer Algorithm
Watson et al. [133] proposed a completely decentralised algorithm for embodied evolution (EE). In their Probabilistic Gene Transfer Algorithm (PGTA), robots exchange randomly selected genes through short range communication. This algorithm doesn’t have a dedicated variation and replacement steps. Each robot holds a single genome of which only individual genes are replaced at runtime. The fact that all genomes are active makes this evaluation inherently parallel. Transmission frequency and gene acceptance are based on the explicit fitness value of the respective robot, which reflects its performance on a task. PGTA has successfully been applied to a photo-taxis task, and they found that it produced better performing solutions than their best hand-designed solutions.
2.4.2 odNEAT
Following the success of NEAT [117], Silvaet al. [112] extended the algorithm for use in a decentralised multi-robot application. They evolved the structure and synaptic
2.4 Algorithms for Distributed Online Evolution 17
weights of the neural network, thus adjusting the level of complexity needing to be specified a priori. The algorithm requires synchronisation of the internal clocks to allow for the use of timestamps. Similar to NEAT, odNEAT uses timestamps within the genotypes to mark the addition of elements to the network to allow crossover. To date, it has been used in simulation and real robot [109] experiments with a number of tasks, e.g. obstacle avoidance, aggregation, maze navigation and homing [111]. It has been demonstrated that controllers across the swarm displayed a diverse range of behaviours and strategies for exploration and aggregation, as well as the ability to adapt to periodic changes in task requirements.
2.4.3 mEDEA
The completely distributed evolutionary algorithm for environment-driven evolution, mEDEA, was first proposed in Bredeche and Montanier [19] and has been used [21,14,
82,80] and adapted [54,95,40,49] many times. The algorithm was demonstrated to be both efficient with regard to providing distributed evolutionary adaptation in unknown environments, and robust to unpredicted changes in the environment.
mEDEA relies on an implicit fitness function that results from two potentially conflicting motivations for a robot: anextrinsic motivation to cope with environmental constraints in order to maximise survival ability and anintrinsic motivation to spread its genome across the population in order to survive. A complex trade-off exists in which behaviours that maximise mating opportunities might negatively impact survival efficiency, e.g. failing to maintain stable energy levels; as a result, mEDEA (or an environment-driven EA) must find some equilibrium between the two states.
The original version of mEDEA exploited a simple strategy in which a robot continuously broadcasts its genome — this can be received and stored by any robot within communication range. At the end of a generation, each robot makes arandom selection from its set of stored genomes, applies a mutation operator, and then replaces
2.4 Algorithms for Distributed Online Evolution 18
its current genome, exactly as in a (1,1) Evolution Strategy [8]. Although there is no selection pressure on an individual basis, from a global perspective, the most widely spread genomes will, on average, be selected more often. While this achieved success in evolving stable populations, it is of interest to attempt to improve both the size of the swarms maintained and their net energy levels to allow for more complex user-defined tasks to be added in future. It is reasonable to assume that spare energy, over and above that required to survive, can be exploited to achieve complex tasks, while a large swarm offers more potential in terms of the tasks that might be accomplished.
Within mEDEA, the evolutionary mechanism differs from natural evolution that also drives adaptation. All robots broadcast their genome continuously and within a fixed range, each robot has equal opportunity to pass on its genome, regardless of its quality. This might create a bias for agents to stay close in order to spread their genome. The reproduction mechanism is asexual, as can be found in bacteria, which uses no form or crossover — variation is provided only by a mutation operator — hence the emphasis is on the spreading ofgenomes rather than genes.
When mEDEA was first proposed in [19] the system was tested under two scenarios: the first evaluated mEDEA in an environment providing limited pressure in which energy is ignored and an agent survives as long as it collects at least one genome. In the second, environmental pressure is introduced by forcing robots to compete for limited resources in order to gain energy. The algorithm was demonstrated to be both efficient with regards to providing distributed evolutionary adaptation in unknown environments, and robust to unpredicted changes in the environment.
Furthermore, given its lightweight nature, mEDEA was demonstrated to be suitable for hardware and software setups that have limited computation. Bredeche et al. [21] implemented mEDEA on a set of E-Puck robots with Linux extension boards to investigate the emergence of consensus amongst the swarm. This was achieved by placing an object in the environment and giving robots a virtual sensor for the object’s location. As there was no benefit or penalty for interacting with the object, the