The contributions are methodological. In specific, new efficient configurators are introduced to tune algorithm parameters. This Section briefly describes each contribution, the motivation behind it, and a summary of its experimental assessment results.
1.3.1. Meta-Optimization with a Flexible Budget
The first contribution concerns offline tuning, where the configurator modifies its solutions to reach better ones. Such configurators are also known as meta-optimization
methods. Meta-optimizers are computationally intensive as evaluating a single parameter setting requires a complete run of the target algorithm on the optimization problem. Also, several replications, for both levels, may be required for stochastic algorithms. Matters become worse if the computational budget for the target algorithm varies, because this will require a re-run of the whole meta-optimizer for each potential new budget. Figure 1.4 displays the performance of an algorithm, applied to a minimization problem, under three different parameter settings. It is clear that the best parameter setting depends on the length of the lower level run.
Figure 1.4. The performance of an algorithm, applied to a minimization problem, under three different parameter settings, each is suitable for a different computational budget.
This work proposes a new meta-optimizer capable of finding the best parameter settings for any computational budget less than a specified maximum in a single run, thus saving a lot of time. The algorithm is called Meta-Optimization with a Flexible Budget, or
Flexible Budget for short. It makes use of the entire convergence curve of a parameter
setting to calculate a rank-based utility.
It will be shown later that, for the experiments conducted here, the Flexible Budget method always finds the best parameter settings for any computational budget less than a specified maximum, while saving about 60%-70% of the computational effort required by the repeated application of the Fixed Budget method, without compromising solution quality. However, some experiments with a small budget using the quad function (defined later), showed minor solution quality degradation. Further details are in Chapter 4. Initial results of this method were presented in Branke and Elomari (2012).
A real-world application of the Flexible Budget is found in shipping. For next-day parcel deliveries, the daily work load varies, which, in turn, affects the time needed to prepare the parcels (e.g. inspection, weighing, sorting.). Such variability disturbs the running time available to a routing algorithm dispatching vehicles on the next day. Obviously, with only one to two hours at hand, the algorithm may perform better with a greedy parameter
0 50 100 150 200 250 300 350 400 450 0 20 40 60 80 100 120 140 160 180 200 S o lu ti o n q u a li ty Iteration
1.3.2. Racing with a self-adaptive significance level
The second contribution also falls under offline tuning where the configurator
selects the best out of a set of parameter settings. There are a number of algorithms that
specialize in efficiently allocating a computational budget (e.g. a training set) among several competing systems, the characteristics of which are unknown beforehand, such that the true best is selected when the budget is consumed. These algorithms guarantee a correct selection in the limit (i.e. if the budget is infinite), and converge to it at varying rates.
The term “system” is originally used in the Simulation Optimization literature to refer to the performance of a simulated system (e.g. a production plant), which is usually modeled with a probability distribution. Here, the term will be used to speak of the performance of an algorithm, or a parameter setting over various problem instances.
Racing is one such algorithm. It works by discarding inferior systems, or parameter settings, as soon as there is enough statistical evidence that they are significantly worse than the current best. In every iteration, the surviving systems are sampled once and the tests are run again. This continues until one system remains, or if the entire budget is consumed, in which case the system with the best performance measure is chosen as the winner. The algorithm’s performance (e.g. the probability of selecting the best) is affected by the significance level α set by the user. A very high α results in less conservative tests and increases the probability of discarding the true best as the systems are dropped off more quickly. A very low α means more conservative tests, where the systems are hardly discarded and the algorithm samples all systems almost equally.
Note that Racing may terminate before the entire budget is consumed. In situations where there is a fixed budget constraint, such that there is no advantage of terminating the algorithm beforehand, a new Racing algorithm, Racing with reset, is introduced. Its basic idea is as follows: whenever a winner is identified and the budget is not entirely consumed, Racing with reset rolls back to the iteration where the first dropout occurred, lowers the
significance level, and runs the tests on all systems. Consequently, if the best system was incorrectly discarded before, it now has another chance to survive in the race. The process is repeated as many times as needed until the budget is consumed. Racing with reset is thus able to consume any given budget and at the same time automatically adaptα.
It will be shown later that Racing with reset, at a relatively highα(say 0.3), always reaches a lower probability of incorrect selection (defined later), than its standard version with the best fixed α set by the user. Moreover, if the variances of the parameter settings’ performance distribution are quite close, or they are highly correlated, its probability of incorrect selection converges to zero faster than one of the best budget allocators from the Simulation Optimization literature, namely the Optimal Computing Budget Allocation (OCBA) algorithm, which addresses a problem similar to the focus of this thesis. See Chen and Lee (2010). There are, of course, situations where Racing with reset is inferior (e.g. if the variances are exponentially increasing), such situations will be detailed in Chapter 5. Initial results of Racing with reset were presented in Branke and Elomari (2013).
1.3.3. One-way Racing with an intelligent budget allocation
All Racing algorithms currently used in the literature rely on a two-way Analysis of Variance (ANOVA) test, so as to account for the effect of the parameter setting (first factor), the effect of the problem instance (second factor), and their interaction. This, in turn, dictates having an equal number of samples for each of the competing systems whenever the tests are run, More importantly, every time a system is discarded, the remaining systems mustall be sampled the same number of times (once is the default). This has two disadvantages, first, it does not allow for a more intelligent way of allocating the budget in each iteration. Second, it does not allow Racing with reset to use all previously collected data if the algorithm terminates before the budget is consumed.
called KW-RaceR and can handle unequal sample sizes. To better allocate the budget in every iteration, instead of sampling all surviving systems equally, OCBA is run to determine the distribution of that iteration’s budget, utilizing all previous knowledge. This combination allows for a different exploration vs. exploitation balance, compared to that obtained with equal allocation (the default). It will be shown that such a combination causes KW-RaceR to perform very similarly to OCBA.