Maximum Loop Iterations

In order to ensure all loops are bounded to terminate within a reasonable evalu- ation time, all indexed loop constructs were restricted to performing a maximum of 100 iterations. This value was arbitrarily chosen and is used consistently on all problems. However, it may be the case that this value is overly restrictive for some problems, or indeed some solutions may actually rely upon this bound to func- tion correctly. Alternatively, this upper bound could be too generous and may be unnecessarily allowing unfit programs to waste valuable evaluation time without any benefit for good program solutions. To test the impact of the maximum loop iterations on the performance of the system, 500 runs were performed on each of five test problems using alternative settings of 50 and 150. Only five of the six test problems are used, because a ForEach loop was used for the even-n-parity problem, which is not constrained by this setting and so the problem is omitted. The values of 50 and 150 were selected as being simple multiples of the original setting and they are also both larger than the largest training and test case index on all problems.

Performance results from these runs are presented in Table 4.17. The results show little variation between the success rates of the three maximum iteration settings on all the problems tested and the overlapping confidence intervals sug- gest that none of the computational effort results are statistically significant. This suggests that the solutions do not have a strong dependency upon the maximum iteration parameter being set specifically at 100, as used in this thesis. However, there are some small variations worth mentioning. In particular, on the sort-list problem the probability of success on the training cases was highest where the iterations parameter was set at 150. Yet the proportion of runs finding a gener- alisable solution was lowest with that same setting. Although these differences are not statistically significant, they do highlight that a potential implication of setting this parameter too high or too low is that the level of generalisation may be reduced.

Table 4.17: Summary of the results comparing different maximum iteration set- tings, as listed in the Its. column. Train% is the percentage of success on the training cases (as used for fitness) and Test% is the percentage of runs that found a solution that generalised to the test set. Effort is the required computational effort to find a solution with 99% confidence and 95% CI is its confidence interval.

Evals is the number of program evaluations required to find a solution with 99%

confidence. The approach used to calculate each of these values is described in detail in section 3.3.

Its. Train% Test% Effort 95% CI Evals

Factorial 50 72.4 72.2 26,300 23,200 - 29,900 526,000 100 72.8 72.0 25,700 22,700 - 29,200 514,000 150 70.4 70.0 27,800 24,600 - 31,500 556,000 Fibonacci 50 59.0 56.6 44,000 37,900 - 51,300 880,000 100 59.0 56.2 40,500 34,800 - 47,400 810,000 150 62.4 60.6 37,700 32,500 - 43,900 754,000 Reverse 50 78.2 76.2 27,700 24,700 - 31,400 138,500 100 78.6 77.0 29,200 25,900 - 33,000 146,000 150 77.6 76.4 28,700 25,600 - 32,300 143,500 Sort 50 73.8 70.4 67,200 57,200 - 79,300 336,000 100 75.0 71.2 65,200 55,900 - 76,300 326,000 150 76.6 69.4 70,400 60,200 - 82,900 352,000 Triangles 50 71.6 71.6 14,400 12,700 - 16,500 86,400 100 69.6 69.6 15,900 13,900 - 18,200 95,400 150 69.8 69.8 14,300 12,600 - 16,400 85,800

Given the comparable performance results, it may be preferable to use a lower maximum iterations setting to potentially reduce evaluation times. Table 4.18 shows the mean fitness evaluation time with each of the three maximum itera- tion settings over these runs. As expected, these results mostly show a positive correlation between the maximum iteration setting and evaluation time, so this justifies an approach of choosing a smaller number of maximum iterations where possible. On these problems, which require repetition to solve, there must be a minimum maximum iterations setting at which the problem is still solvable. It would be interesting to identify this point for each of the problems and test the performance impact of values around this point. This remains for future work.

CHAPTER 4. STRONGLY FORMED GENETIC PROGRAMMING 81

Table 4.18: Comparison of the mean time required to evaluate an individual with maximum iterations settings of 50, 100 and 150

Mean Evaluation Time (ns)

50 Iterations 100 Iterations 150 Iterations

Factorial 79753 ± 32 109592 ± 52 139911 ± 79 Fibonacci 51577 ± 18 63909 ± 29 72372 ± 38 Reverse 81375 ± 41 91035 ± 42 93888 ± 46 Sort 542304 ± 228 724187 ± 333 675877 ± 278 Triangles 16087 ± 23 15823 ± 13 16628 ± 30

4.8

Summary

This chapter has introduced a novel mechanism for adding structural constraints to a tree-based GP representation and has established that these constraints are sufficient to impose a naturally high-level imperative structure upon the evolved programs. It was demonstrated how these programs could be converted to the source code of a number of different high-level imperative programming languages by using code templates. The solutions which were produced to each of the prob- lems were found with high success rates and generalised well to wider test inputs, while using a relatively low amount of computational resources in comparison to those studies reviewed in chapter 3. The reduced size of the search space caused by the additional structural constraints was investigated and shown to be bene- ficial on five out of the six test problems, but there is a warning of the potential to damage success rates in the reduced performance on the reverse-list problem. Finally, the impact of two new control parameters, code-block size and maximum iterations, was explored. The conclusion was that the ideal code-block size is prob- lem dependent but with a preference for a smaller value. The maximum iterations setting was discovered to have little impact on results as long as it is higher than some unknown threshold to solve a given problem, but also that a lower value does in most cases reduce the average evaluation time, as may be expected.

High-Level Imperative Extensions

5.1

Introduction

It was demonstrated in chapter 4 that SFGP provides a general mechanism for constraining the structure of program trees and that those constraints are suffi- cient to evolve programs with an imperative structure using standard high-level imperative programming constructs such as loops and arrays. However, more can be achieved with some additional modifications to the algorithm to specifically target the evolution of these imperative programs. This chapter will propose two such extensions. The first adds the feature of a dynamic syntax, which may be updated by a program to allow new limited scope variables to be declared. The second is a simple method for improving the evaluation performance of programs with the imperative structure, by considering multiple variables as candidates for supplying the return value of a subroutine. The modifications necessary for each of these extensions will be described and some experimental results will be pre- sented that demonstrate their impact, along with a discussion of the potential benefits.

CHAPTER 5. HIGH-LEVEL IMPERATIVE EXTENSIONS 83