Proposed Automatic Algorithm Configuration Method

In this section, the proposed approach for automatically configuring a given algorithm on a per-instance base is introduced as illustrated in Figure 4.1. The major components of the proposed approach: feature extraction, the classification algorithm and the procedures for training and testing are described.

Figure 4.1: The Proposed Automatic Algorithm Configuration Method on a Per- instance Base.

4.2.4.1 Feature Extraction and Classification Algorithm

In the proposed automatic algorithm configuration method, the problem in- stances are represented as a set of aep measures computed based on a set of fitness- probability clouds generated using different neighbourhood operators. The overall goal is to train a classifier to learn the pattern which governs the choice of algorithm configurations, and thus identifying whether a configuration of the target algorithm

is suitable for solving a problem instance. Since the learning is supervised, i.e., learn- ing where a training set of correctly-identified observations is available. In this case, a training sample D consists of an instance’s features and a configuration will be labelled as L ∈ {1, −1}, indicating whether the configuration is suitable for solving the instance or not.

A training sample consists of a problem instance’s features and a configuration of the target algorithm, which is denoted as D = (PF, θ), where PF in D denotes the features of a problem instance and θ denotes a configuration of the target algo- rithm. In the proposed approach, problem instances are represented by a set of aep measures computed based on the fitness-probability clouds generated using different neighbourhood operators.

To determine the label for a training sample, for the target algorithm in a given configuration θ, a number of independent runs on the problem instance need to be performed where a statistical measure (e.g. mean number of the function eval- uations) is taken to determine the algorithm performance. According to a pre- defined threshold on the algorithm performance, the label L is then determined as L ∈ {1, −1}, where 1 represents the configuration of the target algorithm is suitable for the problem instance, and -1 not.

The choice of the classification algorithm is essential to the performance of the proposed automatic algorithm configuration method. Typically the classification algorithm is selected based on the characteristics of the training data. In this case, the size of training samples is relatively small especially at the beginning since repeated executions of the target algorithm in different configurations are needed for each problem instance to generate training samples which is quite time-consuming. Also the training data is expected to be imbalanced since the training samples with the label L = 1 can be much less than those with the label L = −1.

(SVM) [25] is selected as the classification algorithm. SVM has strong theoretical foundations and excellent empirical successes, which is developed on the basis of statistical learning theory and structural risk minimization and can handle small samples, nonlinear, high dimensionality, over learning and local minimum. Support vector machines have been considerably developed in pattern recognition, regression analysis, function estimator and time series prediction [33].

In the binary classification setting, SVM is a non-probabilistic binary linear classifier which takes a set of input data and predicts, for each given input, which of two possible classes forms the output. Given a set of training data {x1, · · · , xn}

that are vectors in some space, and their labels {y1, · · · , yn} where yi ∈ {−1, 1},

an SVM classification algorithm builds a model that assigns new examples into one category or the other. An SVM model is a representation of the examples as points in space, mapped so that the training data are separated by a maximal margin. New examples are then mapped into that same space and predicted to belong to a category according to which side they fall on.

4.2.4.2 Training and Testing Phases

The overall steps in training and testing phases are summarised in Algorithm 2. In the training phase, the features of a problem instance, i.e., a set of aep measures are first obtained. The label attached to a training sample is determined by execut- ing the target algorithm under a given configuration on a given problem instance. This procedure is repeated until the entire training data set is built. Then a classifier is obtained by running the classification algorithm on the training data, which can be used to determine whether a configuration of the target algorithm is suitable for solving a given problem instance. In the testing phase, the classifier takes input a configuration of the target algorithm and features of a given problem instance, i.e., a set of aep measures, and predicts whether the configuration is suitable for solving

Algorithm 2: Procedures for Training and Testing Training Phase

Inputs:

A: the target algorithm; I : a set of training instances; Θt: a set of configurations;

Outputs:

C: A classifier to determine if a given algorithm configuration is suitable for solving a problem instance;

Procedure: begin

Let P F = Features of an instance in I; Let L = {−1, 1}

Let F = a function to evaluate performance of the target algorithm executed under a configuration on an instance;

for inst ∈ I do for θ ∈ Θt do

F = Execute target algorithm A under configuration θ on instance inst;

L = Attach the label to (P F, θ) according to F ; training sample(i) = (P F, θ, L)

i + +; end end

C = Run classification algorithm on the training sample ; Output C;

end

Testing Phase Inputs:

C: the classifier;

inst: an arbitrary instance; θ: a configuration in Θ; Outputs:

L: Performance prediction of the configuration θ on inst; Procedure:

begin

Let P F = Features of inst;

L = Feed (P F , θ) to the classifier C; Output L;

the problem instance or not.

4.2.5

Automatically Configuring (µ + λ) Evolutionary Algo-