Role of Population Size

The above discussion concerned only the choice of the number of generations to be run, given a population size M of 500. We now consider the choice of the population size M. Our experience is that a larger population size M increases the cumulative probability P(M, i) of

satisfying the success predicate of a problem for genetic programming. In the extreme case, if the population is large enough, a solution to the problem can be found at generation 0 by blind random search. However, there is a point after which the cost of a larger population (in terms of individuals to be processed) begins to exceed the benefit obtained from the increase in the cumulative probability of success P(M, i). We proceed by considering the 6-multiplexer problem with a population size of 1,000, 2,000, and 4,000. To simplify the discussion, we do not employ over-selection here; however, we do revisit these same three population sizes with over-selection in section 25.6.

Figure 8.5 shows the performance curves for a population size of 1,000 (without over-selection) for the 6-multiplexer problem and with the function

Page 199 set Fl = {AND, OR, IF, NOT}. The figure is based on 38 runs. The cumulative probability P(M, i) of success is 50% at generation 48 and 53% at generation 50. The numbers 48 and 343,000 in the oval indicate that, if this problem is run through to generation 48, processing a total of 343,000 individuals (i.e., 1,000 x 49 generations x 7 runs) is sufficient to yield a solution of this problem with 99% probability.

Figure 8.6 shows the performance curves for a population size of 2,000 (without over-selection) for the 6-multiplexer problem and with the function set F1 = {AND, OR, IF, NOT}. The figure is based on 148 runs. The cumulative probability P(M, i) of success is 91% at generation 50. The numbers 49 and 200,000 in the oval indicate that if this problem is run through to generation 49, processing a total of 200,000 individuals (i.e., 2,000 x 50 generations x 2 runs) is sufficient to yield a solution of this problem with 99% probability by generation 49.

The cumulative probability of success P(M, i) at generation 49 is 91%. In contrast, just one generation earlier (i.e., at generation 48), P(M, i) is 86%,

Figure 8.5

Performance curves for population size M = 1,000 for the 6-multiplexer problem.

Figure 8.6

Performance curves for population size M = 2,000 for the 6-multiplexer problem.

Page 200 so the number of independent runs R(z) is 3 and 294,000 individuals (i.e., 2,000 x 49 generations x 3 runs) are required. That is, the slopes of the P(M, i) curve and the R(z) function are such that, as P(M, i) rises past the threshold percentage of 89%, R(z) drops from 3 to 2. Thus, by continuing the run for one additional generation, the total number of individuals that must be processed is cut by approximately one-third. Figure 8.7 shows the performance curves for a population size of 4,000 (without over-selection) for the 6-multiplexer problem and with the function set Fl = {AND, OR, IF, NOT}. The cumulative probability of success P(M, i) at generation 39 is 100%. The figure is based on 15 runs. The numbers 39 and 160,000 in the oval indicate that if this problem is run through to generation 39, processing a total of 160,000 (i. e., 4,000 x 40 generations x 1 run) individuals is sufficient to yield a solution of this problem with 99% probability by generation 39.

In contrast, just one generation earlier (i.e., at generation 38), P(M, i) is 93%, so the number of independent runs R(z) is 2 and 312,000 individuals (i.e., 4,000 x 39 generations x 2 runs) are required. That is, the slopes of the P(M, i) curve and the R(z) function are such that, as P (M, i) rises past 99%, R(z) drops from 2 to 1. Thus, by continuing the run for one additional generation, the total number of individuals that must be processed is cut approximately in half.

In summary, a choice of population size of M = 4,000 yields a solution after processing only 160,000 individuals for the 6-multiplexer problem with the function set F₁ = {AND, OR, IF, NOT}. This total of 160,000 is better than the total of 245,000 required for a population size M = 500, the 343,000 for M = 1,000, and the 294,000 for M = 2,000.

Note that the size of the search space (264≈ ₁₀19_{) for the 6-multiplexer problem is very large in relation to the 160,000 individuals that need be} processed using genetic programming for a population size of M = 4,000. Even if we used a less efficient population size, such as 500, 1,000, or 2,000,

Figure 8.7

Performance curves for population size M = 4,000 for the 6-multiplexer problem.

Page 201 the number of individuals that need be processed (i.e., between 245,000 and 343,000) is still very small in relation to the size of the search space.

All of the analysis in this chapter is, of course, retrospective. That is, we started by making numerous runs (successful and unsuccessful) of the problem in order to obtain the instantaneous probabilities of success γ(M, i) by generation. We then computed the cumulative probabilities P(M, i) of success by generation. We then used the cumulative probabilities to determine the number of independent runs R required to yield at least one successful run of the problem with a 99% probability. Finally, we computed the computational effort I(M, i, z) required from the number of independent runs. Computational effort provides a basis for measuring the difficulty of solving a particular problem and a basis for comparing the relative difficulty of solving different problems. This retrospective analysis may be useful in planning future runs if one believes that some new problem is similar in difficulty to a problem for which the performance curves have already been established. In that event, the performance curves may provide some general guidance on the choice of the population size M and the maximum number G of generations to be run for the new problem. The guidance will be especially useful if one believes that the choice of the population size was optimal or near-optimal for the previous problem.