Evo-Devo Systems

Not only evolutionary computing can be utilised to synthesise developmental processes in a more effec- tive way than synthesising them “by hand”, but also, as many believe (for example [17]), developmental systems can also bring evolvability, and scalability, among other features, to evolutionary computing. Moreover, it is much more a bio-plausible solution to evolutionary computing than direct encoding of the phenotype.

Although there is no clear-cut borderline for what is considered to be an evo-devo and what is just an evolutionary algorithm, all the researchers unanimously consider a direct mapping from genotype to phenotype a sign of a non-developmental evolutionary system. Direct encoding aside, Bentley and Kumar [23] differentiated the developmental processes of evo-devo systems into three different classes of external, explicit, and implicit encoding. The first group consist of the indirect mapping that are designed, fixed and not evolved. The developmental process in the other two groups are evolved. Evo-devo systems with explicit encoding are those that evolve a data structure of instructions, which explicitly specify the developmental process. Evo-devo systems with implicit encoding, on the other hand, are those that use a set of rules (instructions) with no or a minimum predefined structure, which can be dynamically activated in parallel based on the context and environmental factors.

In a somewhat similar way, Floreano et. al. [103] classified evo-devo systems in the following four categories:

1. Parametric evo-devo systems with a fixed but nontrivial developmental or generative process that maps a set of evolved parameters from genotype to the phenotype: In this class of evo-devo sys- tems, the developmental process is always fixed and not evolved. This will leave the very difficult problem of synthesising an evolvable and useful developmental process to the designer of the evo-devo system.

2. Evo-devo systems that iteratively execute developmental modifications in a fixed order for a fixed number of times while the modifications that are executed repeatedly are evolved. Parallel rewrit- ing systems are good examples of this category with fixed rewriting cycles that are always exe-

cuted in the same order while the axiom and rewriting rules are defined by the genotype and can be evolved.

3. Evolutionary developmental programs that evolve programs using a set of fixed instructions or functions. Here, the order and interrelation of different instructions can be evolved while the basic instructions are fixed. Cellular Encoding [133, 132], HyperNEAT [349], and using GP and CGP for evolving developmental programs are good examples of this category that use a fixed set of basic functions and terminals but evolve graphs that defines how these basic function work together to develop the phenotype. Applications of Cellular Encoding, GP and CGP in evo-devo neural networks is discussed in more detail in section 2.5.5.

4. Evolutionary developmental processes: These are bio-plausible methods that allow evolving both the basic mechanisms and the general process of the developmental system and cannot be classified in any of the above categories. These are generally more detailed and less abstracted models with higher computational complexities. They generally have the potential for better scalability and evolvability but they also usually require evolving the developmental process from scratch that means much longer runtimes for evolutionary algorithm. However, using already evolved seed populations instead of random initial populations to provide the basic and generic mechanisms of development may mitigate this problem. They also allow for interaction of the environment with the developmental process leading to phenotype plasticity [277] during the lifetime of the individual that can bring other adaptive features such as fault-tolerance, regeneration, and self- repair to the evo-devo systems.

After Bentley and Kumar introduced the term “Embryogeny” [23], Stanley et. al. used the term “Artificial Embryogeny” in [351] referring to evo-devo systems and acknowledged the importance of both evolvability and feasibility of the developmental systems in the evaluation and selection of evo- devo models. They introduced a taxonomy of evo-devo systems based on the following five dimensions that are also very useful in evaluating the bio-plausibility of the evo-devo systems:

1. Cell fate: How many different methods can be used in the system for determination of the role of a cell in the matured phenotype. Systems with few determination methods are residing at one extreme of this dimension and systems with many methods (as in natural evo-devo systems) are at the other end.

2. Targeting: The way that each cell finds other cells that need to be connected (this is particularly important in neurodevelopment as neurons need to connect to each other in a controlled fashion). This can be done by targeting cells that are at a position relative to the source cell or by using specific chemical markers of the target cells. At one extreme of this dimension are systems that only use relative targeting while systems at the other extreme can only use specific targeting. Natural evo-devo and bio-plausible systems reside in the middle as they can use a mixture of both methods.

3. Heterochrony: How the genotype encoding allows evolutionary changes in the timing and order of the events in the development process. At one extreme of this dimension are systems that are not flexible and there is no way to change the steps or order of the developmental events. At the other end are natural evo-devo systems that can change the order or entirely skip some development stages by small changes in the genome.

4. Canalisation: That is evolution of robust developmental mechanisms that are not sensitive to mu- tations and can buffer the effect of mutations. At one extreme of this dimension reside systems with precise developmental processes that are sensitive to the mutations. On the other extreme are natural evo-devo systems that are able to utilise stochasticity, resource allocation [351], overpro- duction/apoptosis, self-regulation, and other means to buffer the effect of the mutations.

5. Complexification: This is the process of evolution starting from simpler phenotypes and incre- mentally add more features and details to the phenotype by using longer genomes. Variable length genomes, neutral gene-duplication mutations that can lead to two genes with different roles in the later generations, synapsis [351], and speciation are crucial for complexification. At one end of this dimension are the systems with fixed-length genomes with no complexification. At the other extremes reside systems with variable length genomes that are able to utilise useful crossovers, synapsis, and speciation.

They also mentioned a few different abstractions in the current models that made artificial evo-devo systems computationally more efficient than bio-accurate models:

• Cartesian coordinates for space (as in HyperNEAT and CGP)

• Using the real time in the developmental process as a regulating mechanism

• Historical marking of the genes to facilitate artificial synapsis (as in NEAT)

• Using a prepared canvas of cells for the development process to start with (as in most of the cell chemistry systems)

They also differentiated these dimensions and abstractions as design decisions from emergent prop- erties and performance measures such as possibility of evolving complex structures, modularity, gene reuse, symmetry, and efficiency. Stanley et. al. [351] also proposed a few benchmarks for evo-devo systems such as evolving symmetry, a specific shape, a specific connectivity pattern, and a simple con- troller.

The classification of the current evo-devo systems by Stanley et. al. in [351] based on the above taxonomy does not include many new models and the ones that are included are not deeply analysed, and their full potentials are not appreciated. Moreover, those five dimensions are not covering all aspects of bio-plausibility of evo-devo systems (e.g. online development). Nevertheless, it clearly shows a few trends in the bio-plausibility of the current models. Their classification shows that although both grammar-based (rewriting systems) and cell chemistry systems are potentially able to close the gap

between the artificial and natural evo-devo systems in most of these five dimensions, the cell grammar- based systems were not very bio-plausible and rich in cell fate and canalisation dimensions; speciation and synapsis are usually neglected in the complexification dimension; grammar-based systems tend to use specific targeting while cell chemistry are more inclined towards relative targeting.

Two major issues in developmental evolutionary systems are evolvability and scalability. Many artificial evolutionary systems suffer from stagnation that limits their evolvability. They get trapped in local minima and can’t find any progressive path to fitter solutions through subsequent mutations. Neutral networks [343] and gradual complexification of the evolving systems [245, 350], and speciation [243] seem to be possible solutions to this problem. Scalability is another emergent property of the developmental processes in nature. Using an example of overhanging blocks, Devert [78] showed that sometimes an explicit iterative development process is necessary for a scalable evo-devo system. Gordon [128] proposed an evo-devo system based on a simple cell chemistry (Outer Totalistic protein rules) for evolution of digital circuits and showed that a developmental process can in fact enhance the scalability of evolutionary digital circuit design.

Another important aspect of the evo-devo systems is the distinction between developed and devel- oping systems. Similar to the difference between evolved and evolving systems, a developed system will eventually mature and stop developing and the genetic code and the developmental process will have no influence on the functioning of the product. On the other hand, in a developing system the developmental process continues to run during the lifetime of the individual that can play a major rule in the adaptivity and maintenance of the system. Roggen et. al. [311] called these systems online developmental systems and suggested that many potentials of evo-devo systems lie in this area. They, however, admitted that using offline development can save computational resources [311].

Backed by such insights, many researchers proposed richer and more bio-plausible models of evo- devo systems. Apart from those evo-devo neural network systems that will be reviewed in more detail in the next section, some of the important and relatively bio-plausible evo-devo systems with general applications are briefly reviewed here.

Liu, Tyrrell, and Miller [223, 224, 261] proposed an evo-devo model, based on cell chemistry using CGP for evolution of the cell dynamics, and applied it first to the french flag benchmark problem [261] and used it later for intrinsic evolution of digital circuits [223, 224]. Their model successfully exhibited fault tolerance to transient damages. In [223] they conjectured that using cell chemistry for long-distance signalling is necessary for achieving robustness in the solutions. Roggen et. al. [311] proposed an online distributed developmental system suitable for multicellular systems in hardware based on a simple cell chemistry and applied it to robust evolution of different types of patterns (uniform, checkerboard, Norwegian Flag, complex CA-generated) to show its scalability and robustness.

Bentley introduced Fractal Gene Regulatory Networks (FGRNs or Fractal Proteins) [25, 24], which uses a fractal-controlled genotype to GRN mapping. It is based on a variable-length genome of genes with two regions. The first region, cis-regulatory site, determines the conditions of the gene expression. The second region, coding region, encodes the protein that can be synthesised by this gene. In Fractal

Proteins system, each protein shape is defined using three real numbers, describing a square subset of the Mandelbrot set. Different protein shapes interact with each other in an artificial chemistry to result in a merged protein shape that will then interact with the cis-regulatory region of each gene (also coded as three real numbers and a couple of thresholds values) to give rise to a gene expression probability. When a gene is expressed, the concentration of the protein encoded in the coding region is increased by a function of the concentration of all proteins interacting with the cis-regulatory region of the gene. The probabilistic nature of the gene expression, complex shapes of the fractal subsets, and non-linear interactions of the protein shapes all seem to contribute to interesting properties of this bio-inspired system. Bentley compared human-designed programs, GP evolutionary programs, and FGRN develop- mental evolutionary programs facing damage and showed that the FGRN developmental program can exhibit graceful degradation [27]. Fractal Gene Regulatory Networks have been successfully applied to function regression and evolving controllers for robots [26, 419], single and joint pole balancing [208], and evolving algorithms for approximating π [207]. Fractal Gene Regulatory Networks [25] showed a high level of bio-plausibility and evolvability [24]. Using very rich domains of fractals for imitating the protein-protein and gene-protein interactions, dynamic indirect mapping of the genotype to the GRN that then controls the dynamics of the cell is much more bio-plausible than other methods using neural networks or GP for control of the cell dynamics.

Apart from cell chemistry and grammar-based developmental systems, there are also generative sys- tems that abstract the explicit process of development and growth through time into a time-independent function. Methods such as CPPN (Compositional Pattern Producing Networks) [347] do not need to sim- ulate all the developmental events through time to generate a pattern in space thus can be classified as an abstraction of the development using a generative process. CPPN-NEAT can be used to evolve a network of mathematical functions with co-ordinates as inputs and structures attributes in that coordinates as out- puts. CPPN has a few advantages over developmental systems: computational efficiency, definition of structures in infinite resolution, perfect damage recovery, and possibility of using user-defined biases. Some adaptive features of online development, such as robustness and regeneration, can be produced by iterating the CPPN so that it reacts to the changes in the environment. However, it lacks a mecha- nism for using local environmental information other than what is already generated by CPPN itself, as demonstrated by Devert in [78]. Some also argue that the sequential nature of an explicit developmental process can provide evolution with useful fitness information during development [203].

A plethora or different evo-devo models for different applications have been proposed by differ- ent researchers. Famous other examples of grammar-based and rewriting evo-devo systems are Jacob’s system for evolution of fractal shapes [174], Kitano’s matrix rewriting systems used for evolution of neu- ral networks [197], Koza’s tree-based rewriting system for evolution of 2D shapes [204], and Hornby’s system for evolving 3D shapes [162]. Among cellular and cell-chemistry evo-devo systems based on diffusion of morphogens, Astor and Adami’s [10] and Hampton and Adami’s [142] models for evolv- ing neural networks, Bentley and Kumar’s Implicit Encoding for evolving 3D shapes [23], Bentley’s Fractal Proteins [25] for evolving GRNs (Gene Regulatory Networks), Bongard and Pfeifer’s Artificial

Ontogeny[48] for evolution of embodied agents, Eggenberger’s differential gene expression system for evolving 3D organisms [88], Kitano’s model of neurogenesis [198], and Miller, Harding and Banzhaf’s Developmental Cartesian Genetic Programming [263, 146] and Self-Modifying CGP [146] used in evo- lution of flags and in many other problems [146] are particularly notable. In the next section focus is on the evo-devo systems used for evolving neural networks.