Recursive Space Subdivision

The method used for visualising program space involves a successive subdivision of a 2D grid, where each subdivision represents the selection of a different codon or symbol. This method is specifically engineered for k-expressions, but it can easily extend to any other abstract syntax tree representation including pointer trees. This method has previously been discussed by Daida et al. [43], who discontinued its use due to a lack of adherence to widely accepted visualisation principles. The technique is admittedly difficult to interpret when trying to compare individual candidates, and plagued by an excess of lines. However, as aninteractivetool (with the ability to scale and translate), the author found it useful to diagnose issues with evolutionary optimisers.

Figure4.23shows an example of what a randomly initialised population of candidate programs represented by Karva-expressions could look like. In this example, a dot represents a single program. The space is divided in a horizontal fashion, for selecting the first symbol, then vertical for the second symbol, and so forth, until all symbols have been selected, at which point a dot is placed. It is worthwhile to note that in doing this, the combinatorial program search space is effectively being viewed as a continuous one (albeit discrete), where differences in programs are represented as spatial differences instead. This idea is essentially equivalent to an edit distance space where symbols towards the root of the expression trees are given a larger weight. In this case, the symbol at the root of thek-expression dictates the largest horizontal row in which the expression falls into.

To further illustrate this method, Algorithms11and12are presented. Algorithm11shows the process by which an expression is added to the tree-based data-structure of the visualiser. Algorithm12is the method by which the data-structure is drawn to the screen. Algorithms11and12are kept separate in the implementation, so that interactive use of the program is more streamlined. The data-structure used is similar in concept to k-D trees, where

98 4. COMBINATORIAL OPTIMISATION

FIGURE4.23: Visual representation of a generation of agents using a tree structure.

space is successively subdivided along each of the principal axes, except not binary division, but space is divided into the number of symbols (terminals and nonterminals).

To further align with visualisation techniques in continuous systems, an indication of movement through space is added here. To indicate candidate movement through this pseudo-space in successive generations, a line is drawn from the previous candidate to the new candidate in each generation. This makes certain dynamics of EAs more clear, particularly theK-GPSO-GPUalgorithm, which is discussed later.

In summary, for a new expression (or program) to be added to the program-space visualisation, the space is first divided intonsections vertically, wherenis the number of terminal and non-terminal symbols. Each section represents a symbol. The first symbol in the expression determines the section next divided. Suppose this is the third section from the top (ie. the third symbol in the entire set of symbols, both terminals and functions). This section is then divided intonsections in a horizontal fashion. The next symbol in the expression determines which section will then be divided further vertically, and so forth. Finally, when no symbols remain in the expression, a dot is drawn to indicate the location of the expression. This allows an entire population of expression trees to be shown in a single image.

The visualiser is best used interactively. Keystroke combinations allow the user to scale the image to magnify specific locations within the program space, and translate the viewport to better understand how the algorithm under scrutiny operates. As seen in Figure4.23, from symbol 4 or 5, it becomes difficult to discern precisely which the final symbols of a program are, but still allows a reasonable comparison among expression trees in a population. The head section of the expression trees tend to be critically important in Karva, and have the ability to greatly change the structure of a tree. This visualiser tends to give top-level symbols more emphasis.

To put this algorithm to the test, a number of visual frames of various algorithms are presented along with a discussion of selected features. In particular, the characteristics of the K-GP-GPU and K-GPSO-GPU algorithms

4.8. PROGRAM-SPACE VISUALISATION 99

withncandidate programs

withpas the top-level symbol drawable setc=p

fori= 0tondo

withmsymbols per program exp = programs[i]

forj= 0tomdo

nextindex = getSymbolIndex(exp[i]) ifp.children.get(nextindex) is nullthen

c=c.addChild(nextindex) c.setLabel(exp[i]) else c=c.children.get(nextindex) end if end for end for

ALGORITHM11: Algorithm for adding ak-expression to the tree data-structure for visualisation by recursive space division.

withmsymbols per program

render(top-level)

FUNCTION render(c) fori= 0tolinecountdo

lines[i].paint() end for

ifchildren is not nullthen

vector2d mystart = getMyStart(); vector2d myend = getMyEnd(); iforientation == Horizontalthen

drawDivisionsHorizontal(mystart,myend) else drawDivisionsVertical(mystart,myend) end if forj= 0to childrencountdo render(children[j]) end for else drawPoint(mycentre) end if END

100 4. COMBINATORIAL OPTIMISATION

described in Sections4.6.1and4.4are compared in terms of convergence for the 3D modified Santa Fe Ant Trail problem. Figure4.24show successive generations of the K-GP-GPU algorithm. These figures show that the K-GP-GPU is very effective at maintaining diversity. How this affects the final result will become more clear when the K-GPSO-GPU is discussed.

FIGURE4.24: Populations of programs in generations 1-4 (top), 5, 10, and 100 (bottom) of a sample run of the K-GP-GPU algorithm, showing good diversity with a steady convergence upon the global optimum.

Figure4.25shows the second frame of a sample generation from the K-GPSO-GPU optimiser. Immediate impressions that this image conveys is the clear use of a global optimum, which is used in crossover (analogous to thegBestparticle). It also indicates that there may be an issue in population diversity, as there is little exploration of the top-level symbols. Instead, it seems that an irreversible loss of information is occurring.

To make this more clear, Figure4.26shows a plot of the unique candidates by generation for the sample run. Having a good number of unique programs is important to ensure adequate diversity for future crossover operations. The difference in diversity by generation for the K-GP-GPU and K-GPSO-GPU algorithms conclusively indicates that there is a serious lack of diversity in the K-GPSO-GPU algorithm. This could potentially cast light on why the GPSO specialised for expression trees is perhaps not as suitable for the problem of inducing expression trees. Togelius et al. in their work proposing Particle Swarm Programming (the amalgamation of GPSO and expression trees) condeded that it is possible thePSOis simply not well suited to the problem domain [280]. The investigation here on diversity seems to support this, even with the Karva expression tree representation. An improvement upon diversity statistics in the K-GPSO-GPU may bring about a better convergence rate.

Observing the scores from the sample generation of the K-GPSO-GPU revealed a large number of the programs obtained a score of zero. Essentially, in the flow of the algorithm with score-weighted crossover, this would result in a replication of the global best. It seemed that what is necessary is a higher mutation rate. Some empirical experiments were carried out to determine whether calibrating this parameter would improve convergence.

Firstly, a much higher mutation rate of0.3 is adopted, (as opposed to 0.1), which did not improve the convergence of the algorithm. The standard deviation of the results was too high to be considered a reliable optimiser. Unique diversity in the population was not maintained, since0.3still seemed too low. The problem with increasing mutation probability further is that the algorithm would fail to converge at all, as the better solutions would almost certainly be mutated to lower fitness values, by the loss of important information.

4.8. PROGRAM-SPACE VISUALISATION 101

FIGURE 4.25: A visualisation of an early generation in the K-GPSO-GPU optimiser, showing an instant convergence to a potentially suboptimal program expression tree from generation 1 to 2.

102 4. COMBINATORIAL OPTIMISATION

Lowering the crossover rate was also experimented with. This was more fruitful, and resulted in a much lower standard deviation among averaged mean fitness values. A crossover probability of0.1seemed to improve the convergence rate. A crossover rate this low does not perform well for genetic algorithms, however. Figure4.27

shows frame2of a sample generation with this modification. In comparison to Figure4.25, what is clear is that most of the population remains stationary.

FIGURE4.27: Visualisation of timestep2of the K-GPSO-GPU algorithm with modified parameters, at this frame,907unique programs are present.

Figure4.28conveys a sense of how the visualiser might respond to human interaction. The top-level program space is shown on the left (generation100of a sample run of K-GPSO-GPU), and successive scaling in on the area where the most candidate programs are quickly indicates the global best candidate without a doubt.