Other Graph-Based Evolutionary Algorithms

In this section we discuss various other graph-based EAs from the literature. This section is not exhaustive; many works could be viewed as relevant and compiling such an exhaustive list would be a research endeavour in itself. Instead, the intention is to give the reader an overview of the varied and extensive use of graphs in evolutionary computation.

Graph-based Genetic Programming. There have been a number of other extensions to GP that utilise a graph-like structure.

Multiple Interactive Outputs in a Single Tree (MIOST) [71, 138] proposes using trees with multiple output nodes and sharing to extend traditional GP to domains where problems have multiple, related outputs. Sharing is created via ‘p’ function nodes, which may point to other nodes in the graph. All other nodes are structured as trees as in conventional TGP. Then traditional tree-based genetic operators (with small modifications to account for p nodes) may be used such as crossover or subtree mutation. While the representation used in MIOST appears to approach that of CGP or PDGP, the use of p nodes as an ad-hoc extension of TGP and the reuse of tree-based genetic operators suggest that this approach is further from

our interests.

Linear-graph GP [114] can be viewed as a graph-based extension of LGP where individuals are represented as graphs. Each node in the graph contains a linear sequence of instructions which are executed when the node is reached. After these instructions are executed, an if- then-else branching instruction is evaluated which selects which node should be evaluated next. Individuals may then be recombined by exchanging linear sequences of instructions between nodes, or by identifying and exchanging entire subgraphs. Experimentally, Linear- graph GP was shown to outperform an LGP system [114].

Tangled Program Graphs (TPGs) [120] is a modular extension of GP whereby nodes in a graph represent cooperating programs which, collectively, constitute an agent capable of interacting with an environment. Each of these nodes is labelled with a sequence of instruc- tions, in the manner of LGP, and therefore the representation of TPGs appears quite similar to Linear-Graph GP. However, as TPGs are generally used to represent agents, it is there- fore often necessary for them to be stateful; it then follows TPGs may contain cycles, unlike Linear-Graph GP. TPGs has been used extensively to learn agents capable of playing video games [120, 121, 210].

Evolution of neural network topology As we have discussed in Sections 2.3.4 and 2.4.3 there are various approaches to neuroevolution which incorporate evolution of topology, such as NEAT [222], ICONE [195] and EANT2 [117]. There are further works which could be considered relevant here.

GeNeralized Acquisition of Recurrent Links (GNARL) [4] instantiates a population of re- current networks by choosing a random number of hidden units chosen from a user-defined range and then adding a random number of connections chosen from a user-defined range3. GNARL primarily modifies networks via mutation, rather than crossover, and distinguishes between weight mutation, which permutes connection weights, and structural mutation. Structural mutations may add or remove hidden units and connections, which is a relatively straightforward process due to the fact that purely RNNs have very few structure constraints. EPNet [264] is a hybrid method which combines structural mutations of topology with backpropagation training of weights. In each iteration of the evolutionary process, network weights are trained by backpropagation. Networks are then replaced with their children,

3

In some sense, initialisation of these networks resembles sampling from a variety of D(n, p) directed random graph models as described in [82].

which are structurally mutated with hidden unit addition/deletion and connection addi- tion/deletion. In some sense, EPNet utilises a form of ‘Lamarckian’ evolution, whereby the population develops (via backpropagation), and their developed features, e.g. connection weights, are passed to the next generation.

Evolution of automata A number of works have investigated the evolution of automata with a view of individual solutions as graphs. In fact, some of the earliest work on evolutionary computation was on the evolution of automata [69]. Some of these works could have been placed in the earlier ‘Graph-Based Genetic Programming’ section as the authors themselves view their contributions as extensions of GP. However, as they evolve domain-specific forms of automata, rather than arbitrary programs, they are described here instead. In the context of the work undertaken here, they are perhaps less useful as points of inspiration as they work with specific forms of automata rather than more generic graph structures, but there is clearly value in examining how they represent and modify graphs.

Graph Structured Program Evolution (GRAPE) [206] combines a graph-like structure with a custom branching functions to evolve automata capable of accessing and modifying an internal register. When a node is evaluated, the register is used in a computation, and then a decision is made over which node should be evaluated next. Special functions are added to the function set which determine branching behaviour based on the contents of the register. GRAPE has been used to induce sorting algorithms [203] and various recursive functions such as factorials [205]. It has also been extended to support Automatically Defined Nodes [204] which take on a similar role as Automatically Defined Functions (ADFs) in TGP [129].

Genetic Network Programming (GNP) [118] proposes an EA over graph-like automata consisting of ‘judgement’ nodes (which operate as if-then-else statements) and ‘processing’ nodes which perform some action. Mutation in GNP is quite similar to that of Recurrent CGP [241], in that with a certain probability, each edge is redirected to point anywhere in the structure. Crossover in GNP bears some resemblance to early work on uniform crossover in CGP [154], where nodes are selected from each parent independently of the rest of the structure. GNP is often used in a reinforcement learning context, to induce agents capable of reading and interacting with some environment; see for example its application to elevator controls [100] and stock-market trading [37].

Parallel Algorithm Discovery and Orchestration (PADO) [231, 232] proposes an EA over graph-like automata which are very similar to those used in GNP. In PADO, nodes have both functional behaviour and branching behaviour, both of which are governed by a stack and

indexed memory. The main application of PADO is to synthesise object recognition systems. Various works in the literature have considered the evolution of Turing machines e.g. [144, 167, 230], although many of these fall under the second constraint we placed on relevant literature. A particular work of interest is found in [177], where the authors represent Turing machines as graphs encoded in a linear genome and develop a crossover operator based on the structure of the underlying graph. Here, crossover is achieved by picking a node in each graph as a crossover point and then exchanging subgraphs reachable within a certain number of connections from the crossover points.

Evolution of Bayesian networks Various approaches have been set out that target the optimisation of Bayesian network structure. For a more detailed review and discussion, see [133]. A number of these approaches focus on modifying the connectivity matrix of a Bayesian network via a GA [66, 134, 148], which, while a valid representation of a graph, is perhaps less interesting in our context. However, several works directly modify the Bayesian network as a graph and therefore are of interest.

In [261], Bayesian networks are treated as DAGs. In each iteration of the EA, offspring are produced by mutation. Mutation operators include edge addition/deletion, edge rever- sal, edge relocation and a knowledge-guided mutation which adds and deletes edges based on minimising the Minimum Description Length (MDL). If an offspring is produced which contains a cycle, it is corrected by deleting the set of edges which induce that cycle.

Bayesian networks have been also been evolved as Completed Partially Directed Acyclic Graphs (CPDAGs) [166]. In this circumstance, individuals in the population represent not only one solution but entire equivalence classes of solutions. Mutation operators may insert and delete both directed and undirected edges, and reverse directed edges. A special mutation operator used appears to resemble a rewrite rule, where a structure of the form X − Y − Z is rewritten to X → Y ← Z. In this work, acyclicity is maintained by checking if a mutation would introduce a cycle before it is applied, rather than the correction process used in [261]. Similarly, [47] proposes a variety of EAs learning Bayesian networks represented both as CPDAGs and DAGs, also utilising this operator.