Although this study of fitness-based incremental evolution on even-n-parity was fairly thorough, there are still plenty of unanswered questions.
The greatest limitation of this work is that only one problem domain, even-
n-parity, was studied. It is highly likely that the results will vary by problem domain, so it would be interesting to apply this technique to different areas.
It would be beneficial to study how the selection of fitness cases impacts the result. In this work we looked only at selecting the firstxfitness cases, but there is the possibility of choosing randomly or possibly pre-ordering the fitness cases with the idea of “coaching” the system. This is very similar to DSS so there would be benefit in a comparative study of incremental evolution and Gathercole’s DSS. When adjusted generations were used it was hypothesised that there was a peak in terms of the optimal number of fitness cases to choose for the first stage (see section 8.5.3). Especially given the trend towards a greater benefit for harder problems, it would be interesting to study this hypothesis under problems even more difficult than even-7-parity.
Qureshi showed that considering a small number of randomly-selected fitness cases per generation (rather than the full set) produced more general solutions for his pursuit game [98, section 4.7]. Our research did not consider this po- tential advantage and so there is scope for future work to study if fitness-based incremental evolution produces more general solutions.
The addition of ADFs was not able to be well studied as the performance at even-6 and -7 was so low for incremental evolution without ADFs. Experiments with larger population sizes would be required. First it should be confirmed that ADFs do indeed offer an improvement above what can be normally expected. If that is confirmed it would also be interesting to analysehow ADFs facilitate the evolutionary process: is the encapsulation hypothesis correct. From other experi- ments we ran where incremental evolution was not beneficial [112], encapsulation was one of the suggested potential “solutions”. It is possible that ADFs were
more useful for this problem domain than they would be in general. Also, we have not considered the fact that executing individuals with ADFs typically takes much longer (from a “CPU time” perspective) than execution without ADFs.
Jackson has offered another form of fitness-based incremental evolution: pa- rameterless functions where the main function is also evolved. Although his solution to its problem (of an increasingly-difficult-to-evolve main function) was dismissed, his solution is not the only option. Study of the balance between a decreased difficulty of the parameterless functions versus an increased difficulty of the main function deserves attention. We, however, are going to continue further down our current road.
Finally, the question that goes begging in this chapter is “how does per- formance change with more than two stages?” Others have studied such sys- tems [12, 14, 16, 17, 34, 42, 50–53, 58, 81, 89, 112] but thorough analysis has not been done. It is likely that the sheer number of permutations limits the fea- sibility of this, but if it were to be done, others’ results would indicate that longer run lengths may be beneficial [44, 83]. In the next chapter we will con- sider an automated process that produces a dynamic number of stages; we also consider longer run lengths.
Automatic Fitness-Based
Incremental Evolution
In the previous chapter we learnt that manual incremental evolution could some- times be beneficial on the even-n-parity problem so long as the results were considered in terms of adjusted generations. The primary reason was that, under adjusted generations, the cost of failure could be significantly reduced.
In this chapter we offer two novel methods for automating fitness-based incre- mental evolution. These methods outperform the results from the last chapter and, under adjusted generations, even regularly outperform direct evolution.
9.1
Motivation
In the study of manual incremental evolution we showed it was possible to out- perform direct evolution. However, it was difficult to see how this level of perfor- mance might be predicted. We learnt:
• ADFs were beneficial to the incremental evolution process. They offered benefits superior to the benefits they offered to direct evolution.
• To see any advantage, it was clear that one had to accept performance measured in terms of adjusted generations. The use of manual incremen- tal evolution decreased the probability of finding a solution but it also decreased both the cost of failure and the cost of success. If these latter advantages are not to be considered, then manual incremental evolution should be discarded.
• The primary beneficial effect was the decreased cost of failure (a benefit that success effort excels at measuring). This was achieved when the first stage used up the allocated generations with relatively little computational effort. However, the results were confusing as they indicated that increasing the number of fitness cases in the first stage improved performance.
• Finally, performance increased as the problem difficulty increased.
Although those results gave some amount of hope for fitness-based incre- mental evolution, it was still unclear how one might configure the stages. The primary unanswered question was how three (or more) stages would impact per- formance. Secondary to that was how many fitness cases each stage should be allocated. Finally, it was unclear whether the use of incremental evolution would be beneficial.
The ideas in this chapter were motivated by the questions of how many stages one should use and how many fitness cases should be specified. What if the number of stages and number of cases were automatically specified depending on a run’s performance? We could set GP an initial number of fitness cases to solve. If it succeeded then we could automatically move on to something more difficult. If it failed then we could reduce the number of fitness cases and try again.
Two methods spawned from this idea and are introduced in section 9.4. Con- ceptually, they differ only in terms of the strategy on success. If GP solves the initial number of fitness cases then one could either aggressively try to solve the goal (or complete set of fitness cases), or one might try a less-aggressive step somewhere between the last success and the goal. However, it’s important to re- member that generations must be spent at the goal stage; it is not very useful to perpetually aim for half the distance between where you are and the goal—such a technique would guarantee failure.
Finally, we are very pleased to be able to say that, unlike the last chapter, you will see that these two methods produced a number of positive results.
9.2
Related Work
As introduced in the literature review (see section 6.2.7), the concept of “auto- matic incremental evolution” is not new. We are aware of two problem-specific forms that have been offered within genetic programming, both of which manipu- lated the environment (as opposed to the fitness function) to make it progressively more difficult.
The first use was originally published by Faustino Gomez in 1999 [52] and then later as part of his PhD thesis [53]. It was a technique to evolve solutions to the two-pole-balancing problem. There were two poles attached to one cart on rails and the controller’s job was to apply a force at each time-step such that it balanced the poles for 100,000 time-steps (30 simulated minutes). The problem is known to become more difficult as the difference between the two pole lengths decreases.
The system started with the shorter pole just 10% the length of the longer pole. If the task was solved, the shorter pole was automatically lengthened by a specified length. If evolution was unsuccessful then the length of the shorter pole was reduced to a length half-way between the current length and the last successful length. This process was repeated, producing on average 30 stages to reach the goal—where the shorter pole was 80% the length of the longer pole. Direct evolution, in contrast, failed to find a solution even where the shorter pole had a 50% length.
Although Gomez’s implementation of this automatic approach was very suc- cessful, he pointed out it was not entirely novel. Others had used a similar technique, but rather than up to 100% increases in the length of the shorter pole, they used only 1% increases [100, 122] and used up to 220 stages [122].
The second use of automatic incremental evolution in GP was by White- son et al. [119]. They studied the keep-away soccer domain by initially fixing the speed of the opponent to just 10% of the controlled players’ speed. When the average player achieved a specified performance level, then the opponent’s speed was automatically incremented five percentage points and evolution continued. However, Whiteson et al. did not study the impact of their approach.
Mouret et al. used incremental evolution to evolve a wing-flapping robot con- troller [88]. They evolved two wing-beat controllers and then evolved a tail controller. They were surprised by the unintuitive combined performance and concluded that “to raise [the] chances of success, the recourse to some sort of au- tomatic incremental methodology seems mandatory”. They felt their experience showed them “one cannot rely on fundamental principles or empirical knowledge” to break up the goal problem into easier sub-problems.
In contrast to the automatic incremental evolution offered by Gomez and Whiteson et al., the form of incremental evolution used in this chapter is one that is potentially generally applicable. Rather than modifying the environment directly, it modifies the way fitness is calculated and thus focuses evolution on a specific area of the problem, automatically gradually enlarging the specific area.
The ideas in this chapter are somewhat similar to the subset-selection schemes1 Chris Gathercole offered in his PhD thesis [44, chapter 6]. Although not a form of incremental evolution, Gathercole’s schemes involved the selection of only a portion of the potential fitness cases used for testing. He found that the tech- niques produced “results as good as those of standard GP and in much shorter time” (albeit not consistently).
Finally, the use of mutation in this work is similar in motivation to the “burst mutation” scheme suggested by Gomez [53] and the delta-coding strategy by Whitley et al. [121]. The similarities are that mutation was used infrequently— its use triggered only when convergence may have stagnated. Both Gomez and Whitley et al. mutated only the population’s best individual, while we have previously discussed benefits of mutating the entire population [113]. In this chapter we took a similar approach in that any individual in the population was a potential mutation candidate.