Montana suggested that one of the benefits of his STGP system is improved performance due to the potentially reduced search-space, courtesy of the type constraints. If that is the case, then it could be expected that SFGP, which introduces even tighter constraints, would reduce the search space even further. Of course, one of the concerns is that reducing the search-space may reduce the number of solutions within that space, or make them more difficult to locate due to a less smooth or disconnected fitness landscape. It is therefore feasible that performance could be either improved by, or degraded by, the addition of structural constaints. In order to test this experimentally, a further set of 500 runs was performed on each of the same problems as in section 4.5, but with node-type constraints removed. Where possible, identical control parameters were used, as described in Tables 4.8–4.13. However, with all node-type constraints removed, a couple of further modifications are unavoidable:
− Loops without node-type constraints are not able to define that they re- quire a variable as a child and so loop implementations are used where their variable is predefined. One such loop node is added to the syntax for each applicable variable in the syntax. The logic of the loops used are identical.
− Similarly, Assignment nodes are unable to specify that the first argument should be a variable. Instead, SET-VAR-x nodes are supplied for each vari- able in the same manner as Montana [109].
Removing the node-type constraints removes the imperative structure that is imposed, so that any node may appear anywhere within a program tree that its data-type allows. This version of the algorithm is directly equivalent to the basic form of STGP. This change increases the search-space as it allows any node of the correct data-type to appear where previously only a node of that data-type
and a specific node-type was allowed. Therefore, the search-space of SFGP can be
is presented in Table 4.15. For comparison, the results from section 4.5.2, where structural constraints are used, are reproduced here in the rows with ‘SFGP’ listed in the experiment column. Performance curves are displayed in Figure 4.7. Table 4.15: Summary of the results comparing SFGP to a system without node- type constraints, shown in the rows labelled STGP. Train% is the probability of success on the training cases (as used for fitness) and Test% is the percentage of runs that found a solution that generalised to the test set. Effort is the required computational effort to find a solution with 99% confidence and 95% CI is its confidence interval. Evals is the number of program evaluations required to find a solution with 99% confidence. The approach used to calculate each of these values is described in detail in section 3.3.
Exp. Train% Test% Effort 95% CI Evals
Factorial STGP 11.2 11.2 383,000 272,000 - 542,000 7,660,000 SFGP 72.8 72.0 25,700 22,700 - 29,200 514,000 Fibonacci STGP 6.6 6.0 1,360,000 827,000 - 2,249,000 27,200,000 SFGP 59.0 56.2 40,500 34,800 - 47,400 810,00 Parity STGP 20.2 9.0 691,000 438,000 - 1,094,000 5,528,000 SFGP 90.2 80.0 39,500 26,000 - 33,700 236,000 Reverse STGP 99.6 99.6 11,100 9,990 - 12,500 55,500 SFGP 78.6 77.0 29,200 25,900 - 33,000 146,000 Sort STGP 69.0 55.0 115,000 98,900 - 134,000 575,000 SFGP 75.0 71.2 65,200 55,900 - 76,300 326,000 Triangles STGP 31.2 31.2 120,000 93,700 - 153,000 720,000 SFGP 69.6 69.6 15,900 13,900 - 18,200 72,000
It would be unfair to make performance comparisons between SFGP and STGP based on these results, as STGP may well be able to make better use of alternative syntax and perform better with different control parameters. However, it serves to illustrate the potential impact of the reduced search space and some of the advantages of SFGP. Computational effort is significantly lower where node-type constraints are used on five of the test problems, but is significantly higher on one of them, the reverse-list problem. This matches our expectations that the reduced search space may impact performance either positive or negatively, depending on the problem and the available syntax.
One of the key reasons for the better performance where the node-type con- straints were used is that solutions were effectively forced to contain a certain level of complexity. The root structure (from Figure 4.4) that is enforced, ensures that all programs contain at least three statements which are highly likely to contain
CHAPTER 4. STRONGLY FORMED GENETIC PROGRAMMING 75 Generation 1 10 20 30 40 50 0 25 50 75 100 Probability of Success (%) 11 : 383000 0 3066666 6133333 Individuals to be Processed P(M,i) I(M,i,z) (a) Factorial Generation 1 10 20 30 40 50 0 25 50 75 100 Probability of Success (%) 18 : 1360000 0 3833333 7666666 Individuals to be Processed P(M,i) I(M,i,z) (b) Fibonacci Generation 1 10 20 30 40 50 0 25 50 75 100 Probability of Success (%) 11 : 691000 0 3066666 6133333 Individuals to be Processed P(M,i) I(M,i,z) (c) Even-n-parity Generation 1 10 20 30 40 50 0 25 50 75 100 Probability of Success (%) 10 : 11100 0 254000 508000 Individuals to be Processed P(M,i) I(M,i,z) (d) Reverse list Generation 1 10 20 30 40 50 0 25 50 75 100 Probability of Success (%) 20 : 115000 0 3066666 6133333 Individuals to be Processed P(M,i) I(M,i,z)
(e) Sort list
Generation 1 10 20 30 40 50 0 25 50 75 100 Probability of Success (%) 7 : 120000 0 382000 764000 Individuals to be Processed P(M,i) I(M,i,z) (f) Triangles
Figure 4.7: Performance curves for each of the test problems, where structural constraints are omitted. P (M, i) is the success rate and I(M, i, z) is the number of individuals to process to find a solution with 99% confidence.
some degree of program logic. In contrast, the populations where only data-type constraints were used, contained many solutions that were comprised of very little complexity and they were unable to make proper use of the constructs available. In this situation, resources are wasted on non-sensical programs that are unlikely to contribute anything towards a correct solution. The reverse-list and sort-list problems suffer much less from this issue, because on these problems the data-type constraints are nearly sufficient to enforce the same root structure. The syntax for each of these problems contains only two nodes which are able to return a value of the correct data-type required for the problem. One of those is the input array variable, which will be unchanged and so any programs using this as the root node will receive a very poor fitness score. The other is SubRoutine, which would result in the same root structure as enforced by the node-type consraints. In our experiments, the proportion of individuals in the population using the input array as the root node was approximately half after initialisation, but on most runs it had dropped to less than 2% of the population after just 3 generations. From this point, the runs can progress in a very similar manner to those where the node-type constraints are used, so the performance does not differ as substantially as on the other problems.
In most cases the level of generalisation has been maintained, but notably the even-n-parity problem shows a substantial drop, with less than 50% of the runs that solved the training cases also solving the test cases. This may be a result of the removal of the enforced nodes in the first couple of levels of the program trees (SubRoutines and CodeBlock). This allows a greater depth of nested boolean expressions to be used, which are then capable of expressing a solution to the even-3-parity problem which is used for training, without any use of iteration. It seems likely that this would be less of a problem if a greater range of training inputs were used. This is a suggestion as to why use of just the 3-bit inputs are sufficient with SFGP; because there is barely sufficient depth available for a non-iterative solution. Had a greater maximum depth been used for the SFGP experiments, our results may have suffered.
CHAPTER 4. STRONGLY FORMED GENETIC PROGRAMMING 77