Evo-Devo Neural Networks

For tackling the challenging task of evolving complex and scalable neural networks and artificial brains many researchers turned to evo-devo systems. Therefore a major part of the evo-devo systems are either designed for evolving neural networks or have been applied to this problem. The neural network evo- devo systems can be categorised into four different types of neurodevelopment: Abstract, Parametric, Explicit, and Implicit.

Abstract Neurodevelopment

The first and most abstracted model of evo-devo NN Systems use an implicit development process that abstracts out the iterative, time-dependent, distributed and local processes of development replacing it with an evolvable function of space (and sometimes time and local variables) describing the connectivity, parameters, and other properties of the neurons and the network, based on their distribution in space. HyperNEAT [349] and its extensions (Adaptive HyperNEAT [304], ES-HyperNEAT [305], HyperNEAT- LEO [388]) are representatives and the state of the art techniques in this group.

HyperNEAT uses NEAT with its historic marking of genes for synapsis, speciation and gradual complexification to evolve CPPNs (Compositional Pattern Producing Networks) [347] with 4 inputs as x,y coordinates of the pre and post-synaptic neurons. Each output of the CPPN can then give the weight of a synapse between two neurons in each layer of the network. Adaptive HyperNEAT [304] allows to evolve NN learning rules by using CPPNs as an evolvable function of previous synaptic weight, neuron locations, and activities. This shows how HyperNEAT can be extended to use local data to abstract local developmental processes using a function. However, as the authors noted, there is still a tradeoff between the generality of the model and the computation cost [304].

Instead of leaving it to the user to specify the locations of the neurons in the substrate (as in Hyper- NEAT), Evolvable-Substrate HyperNEAT (ES-HyperNEAT [305]) distribute the neurons based on the variance of the synaptic weight CPPN function. In HyperNEAT the expression of links (existence of synapses) between neurons is specified by a threshold on the weight, while in HyperNEAT with Link Expression Output (HyperNEAT-LEO [388]) a separate CPPN, which can be seeded with a biased to- wards modularity with local connections, controls the expressions of the links. All of these extensions of HyperNEAT can improve the performance, evolvability and scalability of the system for evolving NNs. A set of experiments showed that ES-HyperNEAT and ES-HyperNEAT-LEO can facilitate the evolution of modular and multimodal (for more than one task) NNs , outperforms the original HyperNEAT, and allows complexification of the NNs as well as CPPNs through evolution [306].

Although one of the main points of abstracting development into a CPPN was to avoid the computa- tional complexity of the explicit iterative developmental processes, the practical, scalable, evolvable and

somewhat more bio-plausible versions of this technique such as ES-HyperNEAT-LEO add extra compu- tation to the development process that reduces the computation cost gap between these techniques and explicit cell-chemistry methods. This group of evo-devo NNs rely on complex mathematical functions and assume that any connectivity is feasible. They are particularly targeted for software-based imple- mentations of the neural or developmental processes and their hardware-based implementations may prove impractical.

Parametric or External Neurodevelopment

The second group are systems based on parametric development [103] or what Bentley and Kumar call External Embryogeny [23]. In these systems the process of neurodevelopment is not evolved and only the parameters that are fed into a hand crafted generative process are evolved. Manual synthesis of evolvable and scalable phenotype-genotype mappings for this group of evo-devo NNs are very difficult and most of the researchers avoid it when it comes to complex neural network systems. The parametric model for generation of NNs by Harp et. al. [148] is an example of such systems. A few more examples can be found in [417].

Explicit Neurodevelopment

The third group of evo-devo systems use a program that is encoded in a data structure and is run explicitly to generate and modify the NN. Bentley and Kumar classify these systems as Explicit Embryogenies [23]. Floreano et. al. divide this group into two subgroups [103]: The first subgroup include those systems that use a grammar-based rewriting system with a fixed order and number of operations while the basic operations (rewriting rules) can be evolved. The final product of the parallel rewriting cycles represent the NN connectivity and parameters. Kitano’s Matrix Rewriting system for evolution of NNs [197] is a good example of such systems. The second subgroup includes those systems that evolve the program (that might also use rewriting rules, self-modification, recursion, or other techniques to achieve modularity), which in turn generates or modifies the NN using a fixed set of operations. Cellular Neural Encoding by Gruau [132] is a good example of such systems.

Implicit Neurodevelopment

Another group of evo-devo NN systems are based on cell-chemistry, morphogens, and evolution of GRNs or other similar dynamical systems that govern the development of the NNs. These systems are what Bentley and Kumar call Implicit Embryogenies [23].

Evo-devo NN systems by Cangelosi, Nolfi, and Parisi [56], Kitano [198], Dellaert and Beer [76], Eggenberger [87], Bongard and Pfeifer [48], and Jakobi [177] can all be considered examples of this type. Astor, Hampton and Adami also proposed a new developmental model for evolution of robust neural networks and reviewed some other developmental neural networks of this kind [10, 142]. All these models are relatively more similar to biology compared to the former types of evo-devo NNs in one way or the other. However, they use different levels of abstraction and complexity, show different performances, evolvability, scalability, and robustness, with different computation costs and they can not be easily compared due to very diverse applications and benchmarks researchers used to evaluate

these models. As a general trend it is clear that more detail (complexity) and lower levels of abstraction always increase the computation cost but not always lead to desired emergent properties or improved performance. Seeking modularity, allowing complexification (of genotype, phenotype, and fitness func- tion), exploiting neutral mutations in the genotype-phenotype mapping, distributed local processes and interactions, and utilising useful environmental and network activity information are examples of good tips for an evo-devo NN designer.

Online Neurodevelopment

Online development has many advantages over using development as a generative process only to create a mature solution and then stoping it. Fault-tolerance, regeneration and self-repair, adaptability, effective use of fitness values during the development, and utilising the activity of the neurons and synapses to guide neurodevelopment are a few examples. Federici [95] used a recurrent neural network to control the cell dynamics for development of neural networks and showed that development can bring regeneration and fault tolerance to spiking neural network robot controllers.

Khan, Miller, and Halliday [191, 192] proposed using online development of CGP programs to evolve both electrical and developmental behaviour of the bio-plausible NNs and showed very interesting bio-plausible emergent properties in their results. This work is particularly unique in the sense that it takes into account different bio-plausible properties and behaviours of the biological neurons (such as nonlinear synaptic interactions and neuron health) instead of limiting the development to modification of synaptic weights with linear summative interactions.

Theoretically, many of the methods mentioned earlier can be also adapted to support online devel- opment. However, the computational cost of continuous running of the development process through the life time of the product is usually prohibitive and needs special attention.

Bio-plausibility in Evo-devo NNs

In [192], Khan, Miller, and Halliday argued that there are much more bio-plausible features than synaptic weights and connectivity that need to be included in evo-devo NNs. They also showed that by bringing many bio-plausible details using their method it is possible to control an agent playing arcade game with only a single neuron [190]. Also, Rust et al. claimed that to exploit the potential of artificial neural systems to the fullest extent, more bio-plausible details of the neural development should be modelled in the artificial developmental systems [321].

There are many other systems that are either the basis of the aforementioned newer models or similar. Reviews of many other evo-devo NN systems can be found in [417, 351, 55, 311, 310, 101, 103]. Among them are those that are designed or suited for hardware implementations, reviewed in the following section.