Simulation Setup

One of the most important elements in setting up the study simulations was un- derstanding how to apply the search algorithms to the task of ANN training. The search algorithm search space consisted of nodal weights in the overall ANN configuration, meaning that each dimension represented a single nodal weight. In the context of this study, an ANN configuration refers to a complete set of nodal weights that creates an ANN. In the BBFWA and PSO algorithms, each particle or spark represented a single ANN configuration. This means that both algorithms effectively train a number of ANN equal to the size of their populations in this study. The CPSO algorithm splits an overall ANN configuration into parts and assigns each part a sub-swarm. While each portion of the ANN configuration has a number of different versions equal to the number of particles in its assigned sub-swarm, the CPSO only trains one full ANN configuration comprised of the best solutions of each sub-swarm.

The CPSO algorithm required a more detailed particle swarm initialization scheme than the BBFWA and PSO algorithms in setting up its initialization process. Determining the best solution for the BBFWA and PSO algorithms consisted of evaluating each particle with Equation 3.3. However, with the CPSO sub-swarm structure, determining the best overall ANN configuration requires combining each sub-swarm particle with particles from the other sub-swarm(s) in order to use Equation 3.3. In this study, particles were combined by index during this process. This means that particle 1 from sub-swarm A would be

evalauted with particle 1 from sub-swarm B, particle 2 from A with particle 2 from B, and so on until all particle pairs were evaluated. Since all CPSO sub-swarms had the same number of particles in them, this scheme worked fairly well at initializing the CPSO particles. This scheme was adopted to reduce the computational time and complexity of evaluating each sub-swarm particle with all other sub-swarm particles, a process that could become prohibitively expensive and time-consuming with larger particle populations and/or more sub-swarms.

3.3.2 Search Algorithm Parameters

The simulations in this study were run using MATLAB R2018b. This study augments research run by Gilley and Yan in a paper that compared search algorithms in improving upon a BP trained ANN [19]. Specifically, this study tests the BBFWA with a different set of parameters in an attempt to improve upon noted weaknesses with the original BBFWA parameters in [19]. [19] used a reduced set of sparks in the BBFWA that put the algorithm at a disadvantage to the other search algorithms. This study used an increased spark count and changed the cr coefficient to attempt to further improve upon the performance of the BBFWA. For the purposes of this study, the BBFWA parameter set from [19] will be referred to as BBFWA Set 1, and the adjusted BBFWA parameters used in this study will be referred to as BBFWA Set 2. The parameters for each algorithm were set as follows:

1. BBFWA Parameter Set 1: ˆ Number of Sparks: 30

ˆ Amplification Coefficient ca: 1.2 ˆ Reduction Coefficient cr: 0.5 2. BBFWA Parameter Set 2:

ˆ Number of Sparks: 50

ˆ Reduction Coefficient cr: 0.25 3. PSO Parameters: ˆ Swarm Size: 30 ˆ c1 Weight: 1.325 ˆ c2 Weight: 1.325 ˆ ω Weight: 0.7298 4. CPSO Parameters: ˆ Number of Sub-Swarms: 2 ˆ Swarm Size: 30 ˆ c1 Weight: 1.325 ˆ c2 Weight: 1.325 ˆ ω Weight: 0.7298

Some of the parameters for the BBFWA were sourced from established literature, and others were determined experimentally. The ca value of 1.2 was sourced from the BBFWA proposal paper [9]. The cr values in both BBFWA parameter sets were both experimentally set. In developing the BBFWA implementation for this study, the cr coefficient was modu- lated in an attempt to improve upon the performance of the algorithm. The crvalue of 0.5 in the first BBFWA parameter set was observed to result in more accurate ANN classification rates than higher cr values. The cr coefficient was then set at 0.25 for the second parameter set because it encouraged better fitness values from the BBFWA, which resulted in more reliable ANN training results. The spark count of 30 in the first set of BBFWA parameters was selected based on reducing the time per training iteration while allowing the BBFWA to still converge to a solution. It was increased to 50 in the second set of BBFWA parameters because the data recorded in [19] showed that a BBFWA spark count of 30 was too low for reliable performance.

The PSO and CPSO shared the same parameters, as the CPSO algorithm uses the same method as the PSO algorithm in the particle training process. The ω inertial weight of 0.7298 was sourced from a PSO hyperparameter configuration proposed by Clerc [20]. Originally, the c1 and c2 weights were experimentally determined by the same process as the BBFWA cr coefficient was. Setting the c1 and c2 weights to 1.325 encouraged better ANN performance on the image data sets. As the Esplit architecture was used for the CPSO algorithm and there were only three layers in the ANN structures in this study, only two sub-swarms were used in the CPSO algorithm. The number of CPSO sub-swarms used in this study mirrors the same design decision made in the CPSO proposal paper, which used 2 sub-swarms with the Esplit architecture on the tested three layer ANN [4].