Initialisation Strategies

In the simple example of Section 2.2.1 we chose to initialise the HM randomly from the entire range of possible values for each component. This works well enough when the search space is small and real time performance is not required. However, there are much better ways to initialise the HM when the designer has some knowledge of the problem and knows where it is likely that good solutions can be found. As we will see in the sections dealing with applications, this is often the case.

Harmony search uses the solutions in the HM as the basis for new improvisations which means that the areas in the search space that harmony search explores first is directly influenced by the initial contents of the HM. As we saw in Section 2.3, the probability of choosing some values over others for the next improvisation is directly controlled by the prevalence of that number in the HM.

Therefore, if one knows nothing about the shape of the search space or where one could likely find good solutions, the best way to initialise the HM would be to maximise diversity by randomly choosing solutions from a uniform distribution over the entire search space. Each random solution then becomes a seed around which harmony search will try to find better solutions. This is essentially an unbiased search through the whole search space. The expectation is that as the solutions move closer together through improvisation and HM updates that at least one of them will pass through the global optimum. When the search space is very large this approach can take a very long time to find the global optima and is usually only used as a last resort. A pseudo code implementation of random initialisation is given in Algorithm 2.2.

input : The uninitialised HM

input :d= the number of components in the solution vector

output: The initialised HM

1 foreachi∈[1,2, . . . ,HMS]do // loop through the vectors of the HM // s is an empty vector of length d+ 1

2 s←[]

3 foreachj∈[1, d+ 1]do // loop through each component

4 s[j]←r // where rvU(LBi,UBi)

5 end

6 s[d+ 1]←f(s) // f is the objective function

7

8 HM[i]←s 9 end

Algorithm 2.2: Random initialisation

Another approach is to uniformly initialise evenly across the range of each deci- sion variable. Instead of randomly initialising values are chosen evenly spaced across a decision variables range. However, when this approach is applied to each decision variable separately multidimensional clustering can result which defeats the purpose of maximising diversity. A better approach to non-random initialisation is to use mul- tidimensional low discrepancy sequences to ensure that the search space is maximally covered without forming clusters [37]. This initialisation method is used in the AHS al- gorithm (see Section 2.6.4) and has shown to be effective in improving the convergence speed of harmony search based optimisers [14].

2.5 Implementation Details

When a rough estimate of the optimal solution is available this can be used as a seed in the HM and leads to a much better initialisation strategy. In many applications the designer has some prior information from which a likely solution can be estimated. We will see examples of prior information being used in this way in Sections 3.4.1 and 4.3.6. It is important to realise that prior information can only provide a rough estimate and that total reliance on this estimate will likely lead to getting trapped in a local optima. It would therefore be a mistake to initialise all the solution vectors in the HM to be equal to the estimated solution. Since there are then no diversity in the HM convergence to the optimum will be very slow if the estimate is far from the real solution. A better solution is to randomly distribute initial solutions around the rough estimate. We initialise the HM with random solutions from a normal distribution with the mean equal to the rough estimate and the variance proportional to the confidence we have in the estimate. Pseudo code for this technique, called seeded initialisation, is given in Algorithm 2.3. Notice that the real difference between random and seeded initialisation is only the distribution ofr and the origin of that distribution’s parameters.

input : The uninitialised HM and the estimate e

input :d= the number of components in the solution vector

input :σ= error variance of e

output: The initialised HM

1 foreachi∈[1,2, . . . ,HMS]do // loop through the vectors of the HM // s is an empty vector of length d+ 1

2 s←[]

3 foreachj∈[1, d+ 1]do // loop through each component

// s[j] is normally distributed around e[j]

// σ is proportional to the confidence in e

4 s[j]←r // where r vN(e[j], σ)

5 end

6 s[d+ 1]←f(s) // f is the objective function

7 HM[i]←s 8 end

Algorithm 2.3: Seeded initialisation

Seeded initialisation has the advantage of taking advantage of prior information without risking slow convergence should the estimate be far from the true optimum solution. As we will see in Section 3.4, seeded initialisation can also be combined with

premature convergence detection to force harmony search to intentionally get stuck in a specific local optima. This is useful when the objective function does not perfectly describe the best solution to the problem and local optima may turn out to be the best solution.

In some situations the components of the solution vector can be split into distinct parts of the search space. Prior information might be available for some of the compo- nents but not for all. A hybrid initialisation scheme can be used to initialise some of the components using the available prior information while other components, of which nothing is known, are randomly initialised. This ensures that all information that can possibly speed up convergence are used. Prior information can either be used as the centre of a random distribution like we saw in seeded initialisation or in specialised cases some components are all set equal to the same prior. Examples of hybrid initial- isation used in this specialised way is discussed in Section 5.3 and Section 5.4. Pseudo code for hybrid initialisation is shown in Algorithm 2.4.

input : The uninitialised HM and the priorp

input :d= the number of components in the solution vector

output: The initialised HM

1 foreachi∈[1,2, . . . ,HMS]do // loop through the vectors of the HM // s is an empty vector of length d+ 1

2 s←[]

3 foreachj∈[1, d+ 1]do // loop through each component

4 if j=d+ 1then // evaluate s

5 s[j]←f(s) // f is the objective function

6 else if p[j]6= #then // p is a vector of prior values. // p[j] = # indicates no prior value is available

// p can also be used like e from Algorithm 2.3

7 s[j]←p[j] 8 else // build s 9 s[j]←r // where rvU(LBi,UBi) 10 end 11 end 12 HM[i]←s 13 end

Algorithm 2.4:Hybrid initialisation

2.5 Implementation Details

available that can possibly guide harmony search to likely solutions. Random initiali- sation is only recommended when the search space is small and nothing is known about likely solutions prior to optimisation. Seeded initialisation should be used when an es- timate can be made of a likely solution. When confidence in the estimate is highσ can be set small to ensure quick convergence to the solution close to that estimate. When confidence is low,σ should be chosen larger to ensure sufficient diversity to escape the estimate’s basin of attraction, should it be the wrong solution. Hybrid initialisation should be used when a complete estimate is not available but some of the components can be estimated from prior information.