Dispersive flies optimisation

Inspiration was taken from nature to solve the optimisation problems of a search space by using the swarming behaviour of the objects in the search space to find optimal solutions. DFO, first introduced in (Al-Rifaie, 2014), is an algorithm inspired by the swarming behaviour of flies hovering over food sources. Several factors affect the swarming behaviour of flies, including the presence of threat, which disturbs the flies’ convergence to the optimal value. As a result, both the formation and breaking of the swarms are considered in the proposed algorithm (Al-Rifaie, 2014). The position vectors of the flies are defined as:

~

xt_i =xt_i1, xt_i2, . . . , xt_iD, i= 1,2, . . . ,N (3.3)

Wheretis the current time step,Dis the dimension of the problem space and

N is the number of flies (population size).

In the first generation, when t = 0, the ith vector’s dth component is ini- tialised as:

x0_id =xmin,d+r(xmax,d−xmin,d) (3.4)

Whereris a random number drawn from a uniform distribution on the unit intervalU(0,1);x_min andx_max are the lower and upper initialisation bounds of the dth _{dimension, respectively. As a result, each fly in the population is}

randomly initialised with a position in the search space (Al-Rifaie and Aber, 2016).

At each iteration, the position vectors’ components are independently up- dated. This is achieved by considering three different aspects: the compo- nent’s value, the corresponding value in the best neighbouring fly’s vector that has the best fitness, and the corresponding value in the best fly in the whole swarm. The updated equation is described below:

xt_id = xt_i−1 nd +U(0,1)×(x t−1 sd −x t−1 id ) (3.5)

Where xt_i−1

nd is the position value of ~x

t−1

i ’s best neighbouring fly in the d th

dimension at time step t − 1; xt_sd−1 is the swarm’s best fly value in the dth

dimension at time stept−1; and U(0,1)is the uniform distribution between 0 and 1 (Al-Rifaie (2014) and Al-Rifaie and Aber (2016)).

Technically speaking, there are two main components of DFO: the use of social neighbouring network in updating the flies’ position and the flies’ communication with each other about the best results found. Moreover, the disturbance of the swarm plays an important role in DFO as it displaces the flies, leading to the discovery of better positions. Thus, at the update phase a stochastic element is introduced that is known as thedisturbance thresholdor

∆. As a result, if a random number,r, generated from a uniform distribution on the unit interval U(0,1) is less than ∆, each component of the flies’ po- sition vectors are reset. Thedisturbance thresholdprovides a disturbance and helps in avoiding stagnation over a local minima (Al-Rifaie, 2014).

Algorithm 9 summarises the DFO algorithm.

Algorithm 9Dispersive Flies Optimisation

1: whileFunction Evalutions<Evaluations Alloweddo

2: fori= 1 →Ndo 3: ~xi.fitness←f(~xi) 4: ~xs =arg min[f(~xi)], i∈ {1, . . . , N} 5: fori= 1 →Nandi6=sdo 6: ~xin =arg min[f(~xi−1), f(~xi+1)] 7: ford= 1 →Ddo 8: if(u <∆)then

9: xt_id+1 ←x_min,d+u(xmax,d−xmin,d)

10: else 11: xt_id+1 ←xt_i nd +u(x t sd−x t id)

In summary, DFO is a population-based algorithm, originally proposed to search for the optimum value over the continuous search space. Although the algorithm is simple, it has been found that DFO outperforms the standard versions of the well-known PSO, GA, and DE algorithms on an extended set of benchmarks over three performance measures of error, efficiency and reli- ability (Al-Rifaie, 2014). It is shown that DFO is more efficient in 84.62% and more reliable in 90% of the28standard optimisation benchmarks used (Al- Rifaie and Aber, 2016). Not only has theoretical research been carried out on the DFO algorithm, but it has also recently been applied to medical imaging

(Al-Rifaie and Aber, 2016); furthermore, ongoing and current research is be- ing conducted in the fields of image analysis, simulation and gaming (King and Al-Rifaie, 2017), computational aesthetic measurements (Al-Rifaie et al., 2017a), digital arts (Al-Rifaie et al., 2017b), protein folding and more.

A comparison between PSO and DFO

PSO is based on the social behaviour observed in a bird flock. It has a pop- ulation called particles that moves around in the search space based on two vectors: the particle position and the velocity. In addition to the two vectors, each particle has a memory of its personal best position. Each particle move- ment in the search space is a factor of its personal best particle and the global best particle. Originally, PSO was applied to continuous optimisation prob- lems. However, it has been extended further to solve discrete optimisation problems and even multi-objective optimisation problems (Kennedy, 2011).

DFO is based on the swarming behaviour of flies hovering over food sources. The formation of the swarm is affected by the presence of threats which can disturb the flies. Thus, in DFO both the formation of the swarms and the breaking of the swarms is considered in the algorithm. In DFO, the movement of the flies is affected by the best neighbouring fly and the best fly in the whole swarm. The flies will be dispersed from their location de- pending on the disturbance threshold. This is to explore new locations in the search space (Al-Rifaie, 2014).

PSO and DFO have many similarities in that both are continuous, population- based optimisers and a fly or particle’s fitness is evaluated using a fitness function. Also, both have a number of parameters to tune that affect algo- rithm performance when dealing with a certain problem. PSO has four pa- rameters: population size, the acceleration constants,c1,2, and the weightw.

Moreover, in terms of memory, PSO requires more memory to store infor- mation on personal previous best particle vector. Unlike PSO, DFO has only two tunable parameters: the population size and the disturbance threshold. Despite the introduction of the other variants of PSO in an attempt to sim- plify the algorithm such as the bare bone PSO in which the velocity vector is removed, DFO still has less components, and it exhibits better performance when using the three measures of error, efficiency and reliability (see Eq 1, 2 and 3) in (Al-Rifaie, 2014) . The simplicity of DFO in terms of parameters and components allows for further analysis.

There are many more swarm intelligence and population-based algorithms available, but they are not discussed in this section as only GA, ACO, PSO,

and DE as well as a more in-depth introduction to SDS and DFO are within the scope of this thesis. The reason is to explore the possibility of using these two population-based algorithms for solving the class imbalance issue in data mining.

The next section reviews the most commonly used population-based al- gorithms in applications related to class imbalance in data mining.