Development of effective cuckoo search algorithms for optimisation purposes

Full text

(1)COPYRIGHT AND CITATION CONSIDERATIONS FOR THIS THESIS/ DISSERTATION. o Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. o NonCommercial — You may not use the material for commercial purposes.. o ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.. How to cite this thesis Surname, Initial(s). (2012). Title of the thesis or dissertation (Doctoral Thesis / Master’s Dissertation). Johannesburg: University of Johannesburg. Available from: http://hdl.handle.net/102000/0002 (Accessed: 22 August 2017)..

(2) DEVELOPMENT OF EFFECTIVE CUCKOO SEARCH ALGORITHMS FOR OPTIMISATION PURPOSES BY MAHLAKU MARELI A dissertation submitted for the partial fulfilment of the requirements for the degree Doctor of Engineering (Electrical and Electronic Engineering) in the Department of Electrical and Electronic Engineering Science Faculty of Engineering UNIVERSITY OF JOHANNESBURG Supervised by Professor Bhekisipho Twala July 2018.

(3) DECLARATION. I hereby declare that this doctoral thesis is my own work, and it does not contain other people’s work without this being stated; and does not contain my previous work without this being stated, and that the bibliography contains all the literature that I have used in writing the thesis, and that all references refer to this bibliography.. Submitted by. Mahlaku Mareli. Date Signature. Approved by (Supervisor). Prof. Bhekisipho Twala. Date Signature. ii.

(4) ABSTRACT Optimisation, the process of finding either a maximum of a minimum of the problem at hand plays a key role in several disciplines including engineering and science. In this thesis, different Cuckoo Search algorithms are developed for effective optimisation purposes. These algorithms are tested on ten mathematical test functions and then used to optimise a Back-Propagation Neural Network used for short-term electricity load forecasting for South African data, with the focus on the City of Johannesburg. The original Cuckoo Search algorithm is based on random walk step sizes derived from Lévy probability distribution and the switching parameter between local and global random walks is constant. However, other probability distributions like Cauchy, Gaussian and Gamma have also been used and the switching parameter can be changed dynamically. The first contribution of the thesis is the development a new Cuckoo Search algorithm whose random step sizes are derived from Pareto probability distribution function. This new Pareto-based Cuckoo Search algorithm is tested on ten benchmark test functions together with other Cuckoo Search algorithms using step sizes derived from Gaussian, Cauchy, Gamma and Lévy probability density functions. When using the confidence interval analysis, the Lévy-based Cuckoo Search algorithm outperforms the Pareto based Cuckoo. However, confidence interval results are only superior due to only one test function whereby Lévy-based Cuckoo Search performed well. Moreover, the Pareto-based Cuckoo shows superior performance in comparison to the other algorithms, leading in seven test functions out of ten when tested for convergence. The second contribution is the implementation of Cuckoo Search algorithms with dynamically increasing switching parameters between local and random walks. The first improvement done on Cuckoo Search algorithm is the implementation of linear increasing switching parameter, the second is the implementation of power increasing switching parameter and the third improvement is the implementation of exponential increasing switching parameter. When tested on benchmark test functions, the exponentially increasing Cuckoo Search algorithm outperforms the other algorithms by obtaining the longest confidence interval of 4.50566 while the next algorithm (original Cuckoo Search) obtains an interval of 3.9699. Moreover, using convergence plots, both iii.

(5) exponentially increasing and linear increasing Cuckoo Search algorithms equally perform well with each leading by three on ten benchmark test functions. The third contribution is the application of Cuckoo Search algorithms to forecast next hour Johannesburg electricity load demand. The neural network training using data from Johannesburg Citypower (hourly electricity data) and South African Weather Services (hourly weather data like temperature, humidity, wind direction and wind speed), the training graphs confirmed that no overfitting occurred. The Cuckoo Search algorithms are further used to forecast hourly electricity demand for the 19th February 2015. The mean absolute percentage errors for different probability based Cuckoo Search algorithms obtained are as follows; 8.4%, 7.2%, 8.3%, 5.6% and 5.8% for the Lévy, Cauchy, Gaussian, Gamma and Pareto, respectively. For the dynamic changing switching parameters, the following mean absolute percentage errors were obtained; 8.4%, 6.2%, 9.7%, 7.1% and 6.7% for the constant, linear decreasing, linear increasing, power increasing and exponential increasing, respectively.. iv.

(6) ACKNOWLEDGMENTS I would like to thank the Department of Electrical and Electronics Engineering for the opportunity provided to pursue a Doctor of Engineering degree at the University of Johannesburg. I would specifically like to express my gratitude to Professor Bhekisipho Twala for supervising my research work and providing positive feedback and guidance during the preparations of this thesis. The financial and technical support received from Institute for Intelligent Systems at the University of Johannesburg were truly remarkable and highly appreciated. The reviewers of my work also provided valuable information and guidance, they are also acknowledged, City Power and South African Weather Services provided 2012 to 2015 Johannesburg electricity data and weather data, respectively. Their support is acknowledged, and it would have been impossible to run forecasting simulations without this data. Their turn-around time to provide the required data was very short. I would like to thank my wife Lerato, my sons Mareli and Thibeli for their support, love and patience. I would also like to thank my mother, Maneo, ‘my brothers and my sisters, my late father Edward and grandfathers Michael Mareli and Lechaka John Makhoali and late grandmother ‘Makabelo Makhoali. They would be pleased to see me improving my education and their support would be noticed. Dieter Bansemer played a significant role in my education decision-making especially in the field of electronics. I truly acknowledge his support that span over 20 years! Finally, I would like to thank GOD, for giving me vision, strength and incredible support, without which the completion of this research would not have been possible.. v.

(7) NOMENCLATURE ANFIS. Adaptive Neural-Fuzzy Inference. ARMA. Auto-Regression-Moving Average. BP. Back Propagation. CCRF. Continuous Conditional Random Fields. CI. Confidence Interval. CPU. Central Processing Unit. CS. Cuckoo Search. CSCo. Cuckoo Search Algorithm using Constant Switching Parameter. CSEI. Cuckoo Search Algorithm using Exponential Increasing Switching Parameter. CSLD. Cuckoo Search Algorithm using Linear Decreasing Switching Parameter. CSLI. Cuckoo Search Algorithm using Linear Increasing Switching Parameter. CSPI. Cuckoo Search Algorithm using Power Increasing Switching Parameter. DA. Deterministic Annealing. DE. Differential Evolution. FA. Firefly Algorithm. FIR. Fuzzy Inductive Reasoning. FL. Fuzzy-Logic. FNN. Fuzzy Neural Networks. GA. Genetic Algorithm. GB. Gigabits. GDP. Gross Domestic Product. GHz. Giga Hertz vi.

(8) GP. Gaussian Process. GRNN. General Neural Networks. HWT. Holt-Winter Taylor. KE. Kinetic Energy. MAE. Mean Average Error. MAPE. Mean Absolute Percentage Error. MSE. Mean Square Error. MW. Megawatts. NARX. Nonlinear Auto-Regression with Exogenous. NF. Neuro-Fuzzy. NN. Neural Networks. PSO. Particle Swarm Optimisation. RAM. Random Access Memory. RBF. Radial Basis Function. RME. Ratio of Medians-based Estimator. RMS. Root Mean Square. SA. Simulated Annealing. SAWS. South African Weather Services. SEC. Smart Energy Consumption. STLF. Short-Term Load Forecasting. SVM. Support Vector Machines. USD. United States Dollar. WFNN. Wavelet Fuzzy Neural Networks vii.

(9) LIST OF TABLES Table 1: Parameters Description for Supervised learning............................................................ 26 Table 2: Global Minima Values of Test Functions (25 variables/ Cuckoo nests). ...................... 48 Table 3: 95% Confidence Interval for Current Probability Based Cuckoo Search Algorithms .. 49 Table 4: Current Probability Based CS Algorithms Convergence Time ..................................... 49 Table 5: Summary of Current Probability Based CS Algorithms Convergence .......................... 54 Table 6: Current Switching Parameter CS Algorithms Descriptions. .......................................... 54 Table 7: Global Average Minimum Results for Current Algorithms .......................................... 55 Table 8: 95% Confidence Interval (CI) for Current Switching Parament CS Algorithms .......... 55 Table 9: Current Switching Parameter CS Algorithms Convergence .......................................... 61 Table 10: Probability-Based CS Algorithms used in MATLAB Simulation. .............................. 66 Table 11: Probability-based Cuckoo Search Algorithms Convergences ..................................... 66 Table 12: 95% Confidence Intervals (CI) for Probability-Based CS algorithms......................... 67 Table 13: Summary of Probability-Based CS Algorithms Convergences ................................... 67 Table 14: Dynamic Switching Parameters Definition.................................................................. 74 Table 15: Different CS Algorithms based on changing Switching Parameter. ............................ 74 Table 16: Switching Parameter based Cuckoo Search Algorithms Convergences ..................... 75 Table 17: 95% Confidence Interval (CI) for Switching Parameter CS Algorithms..................... 76 Table 18: Summary of CS Algorithms Convergence Leading .................................................... 76 Table 19: South African Seasons ................................................................................................. 86 Table 20: Seasons Correlation Coefficient Between Electricity Load and Weather Parameters. 87 Table 21: Correlation Coefficients for one Year period .............................................................. 88 Table 22: Hourly weather data and units ..................................................................................... 89 Table 23: Neural Network Input Variables .................................................................................. 90 Table 24: Neural Network Output Variable ................................................................................. 90 Table 25: Hourly STLF Regression for Summer Data ................................................................ 92 Table 26: Hourly STLF Regression for Autumn Data ................................................................. 92 Table 27: Hourly STLF Regression for Winter Data ................................................................... 93 viii.

(10) Table 28: Hourly STLF Regression for Spring Data ................................................................... 93 Table 29: STLF Best Validation Performance using Probability Based CS................................ 94 Table 30: Hourly STLF Regression for Summer Data ............................................................. 115 Table 31: Hourly STLF Regression for Autumn Data .............................................................. 115 Table 32: Hourly STLF Regression for Winter Data ................................................................. 115 Table 33: Hourly STLF Regression for Spring Data ................................................................. 116 Table 34: STLF Best Validation Performance using Dynamic Switching Parameter CS ......... 116 Table 35: 2015 February the 19th Load Forecasting using probability-based CS algorithms .... 133 Table 36: Mean Absolute Percentage Errors (MAPE) for probability based Cuckoo Search algorithms ................................................................................................................................... 134 Table 37: 2015 February the 19th Load Forecasting using changing switching parameter CS algorithms ................................................................................................................................... 135 Table 38: Mean Absolute Percentage Errors (MAPE) for using changing switching parameter Cuckoo Search algorithms .......................................................................................................... 136. ix.

(11) LIST OF FIGURES Figure 1: Neuron with connecting weights and bias. ................................................................... 22 Figure 2: Supervised Learning. .................................................................................................... 25 Figure 3: Five algorithms used to train NN [128]. ....................................................................... 28 Figure 4: Unsupervised Learning. ................................................................................................ 29 Figure 5: Reinforced Learning. .................................................................................................... 29 Figure 6: Cuckoo Search Pseudo Code for a Global Minimum Problem. ................................... 35 Figure 7: The Pseudo Code of the Cuckoo Search Back Propagation. ........................................ 40 Figure 8: Average Cuckoo Search algorithms convergences for Ackley function. .................... 50 Figure 9: Average Cuckoo Search algorithms convergences for Griewank function.................. 50 Figure 10: Average Cuckoo Search algorithms convergences for Bohachevsky function. ....... 50 Figure 11: Average Cuckoo Search algorithms convergences for De Jong function. ................. 51 Figure 12: Average Cuckoo Search algorithms convergences for Matyas function.................... 51 Figure 13: Average Cuckoo Search algorithms convergences for Zakharov function. ............... 51 Figure 14: Average Cuckoo Search algorithms convergences for Goldstein-Prices function. .... 52 Figure 15: Average Cuckoo Search algorithms convergences for Rosenbrock function. ........... 52 Figure 16: Average Cuckoo Search algorithms convergences for Easom function. .................. 53 Figure 17: Average Cuckoo Search algorithms convergences for Michalewicz function. .......... 53 Figure 18: CSCo and CSLD convergence for Ackley test function. .......................................... 56 Figure 19: CSCo and CSLD convergence for Griewank test function. ....................................... 56 Figure 20: CSCo and CSLD convergence for Bohachevsky test function. ................................. 57 Figure 21: CSCo and CSLD convergence for De Jong test function........................................... 57 Figure 22: CSCo and CSLD convergence for Matyas test function. ........................................... 58 Figure 23: CSCo and CSLD convergence for Zakharov test function. ....................................... 58 Figure 24: CSCo and CSLD convergence for Goldstein-Prices test function. ............................ 59 Figure 25: CSCo and CSLD convergence for Rosenbrock test function. .................................... 59 Figure 26: CSCo and CSLD convergence for Easom test function. ............................................ 60 Figure 27: CSCo and CSLD convergence for Michalewicz test function. .................................. 60 x.

(12) Figure 28: Proposed Pareto-based Cuckoo Search algorithm flowchart. .................................... 65 Figure 29: Average CS convergences on Ackley function. ......................................................... 68 Figure 30: Average CS convergences on Griewank function. ..................................................... 69 Figure 31: Average CS convergences on Bohachevsky function. ............................................... 69 Figure 32: Average CS convergences on De Jong function. ....................................................... 70 Figure 33: Average CS convergences on Matyas function. ......................................................... 70 Figure 34: Average CS convergences on Zakharov function. ..................................................... 71 Figure 35: Average CS convergences on Goldstein-Price function. ........................................... 71 Figure 36: Average CS convergences on Rosenbrock function. ................................................ 72 Figure 37: Average CS convergences on Easom function. .......................................................... 72 Figure 38: Average CS convergences on Michalewicz function. ............................................... 73 Figure 39: Cuckoo Search algorithms average convergences for Ackley function. ................... 77 Figure 40: Cuckoo Search algorithms average convergences for Griewank function. ................ 78 Figure 41: Cuckoo Search algorithms average convergences for Bohachevsky function. .......... 78 Figure 42: Cuckoo Search algorithms average convergences for De Jong function. ................. 79 Figure 43: Cuckoo Search algorithms average convergences for Matyas function. .................... 79 Figure 44: Cuckoo Search algorithms average convergences for Zakharov function. ............... 80 Figure 45: Cuckoo Search algorithms average convergences for Rosenbrock function. ............ 81 Figure 46: Cuckoo Search algorithms average convergences for Goldstein-Prices function. ..... 81 Figure 47: Cuckoo Search algorithms average convergences for Easom function...................... 82 Figure 48: Cuckoo Search algorithms average convergences for Michalewicz function. .......... 82 Figure 49: Feed-Forward Neural Network with Levenberg-Marquardt Back Propagation......... 90 Figure 50: Levenberg-Marquardt BP NN optimised by Cuckoo Search Algorithm [184]. ......... 91 Figure 51: Current Lévy Cuckoo Search validation using Autumn data. .................................... 94 Figure 52: Current Lévy Cuckoo Search regression using Autumn data. .................................. 95 Figure 53: Current Lévy Cuckoo Search Algorithm validation using Winter data. .................... 95 Figure 54: Current Lévy Cuckoo Search Algorithm Regression using Winter data. .................. 96 Figure 55: Current Lévy Cuckoo Search Algorithm validation using Spring data...................... 96 Figure 56: Current Lévy Cuckoo Search Algorithm Regression using Spring data. ................... 97 xi.

(13) Figure 57: Current Lévy Cuckoo Search Algorithm validation using Summer data. .................. 97 Figure 58: Current Lévy Cuckoo Search Algorithm Regression using Summer data. ................ 98 Figure 59: Current Gaussian Cuckoo Search Algorithm Validation using Autumn data. ........... 98 Figure 60: Current Gaussian Cuckoo Search Algorithm regression using Autumn data. .......... 99 Figure 61: Current Gaussian Cuckoo Search Algorithm Validation using Winter data. ............. 99 Figure 62: Current Gaussian Cuckoo Search Algorithm regression using Winter data. ........... 100 Figure 63: Current Gaussian Cuckoo Search Algorithm Validation using Spring data. ........... 100 Figure 64: Current Gaussian Cuckoo Search Algorithm regression using Spring data. ............ 101 Figure 65: Current Gaussian Cuckoo Search Algorithm Validation Using Summer data......... 101 Figure 66: Current Gaussian Cuckoo Search Algorithm Regression Using Summer data. ....... 102 Figure 67: Current Cauchy Cuckoo Search Algorithm Validation Using Autumn data. .......... 102 Figure 68: Current Cauchy Cuckoo Search Algorithm Regression Using Autumn data........... 103 Figure 69: Current Cauchy Cuckoo Search Algorithm Validation Using Winter data. ............ 103 Figure 70: Current Cauchy Cuckoo Search Algorithm Regression Using Winter data............. 104 Figure 71: Current Cauchy Cuckoo Search Algorithm Validation Using Spring data. ............. 104 Figure 72: Current Cauchy Cuckoo Search Algorithm Regression Using Spring data. ............ 105 Figure 73: Current Cauchy Cuckoo Search Algorithm Validation Using Summer data. ........... 105 Figure 74: Current Cauchy Cuckoo Search Algorithm Regression Using Summer data. ......... 106 Figure 75: Current Gamma Cuckoo Search Algorithm Validation Using Autumn data. .......... 106 Figure 76: Current Gamma Cuckoo Search Algorithm Regression Using Autumn data. ......... 107 Figure 77: Current Gamma Cuckoo Search Algorithm Regression Using Winter data. ........... 107 Figure 78: Current Gamma Cuckoo Search Algorithm Regression Using Winter data. ........... 108 Figure 79: Current Gamma Cuckoo Search Algorithm Validation Using Spring data. ............ 108 Figure 80: Current Gamma Cuckoo Search Algorithm Regression Using Spring data............. 109 Figure 81: Current Gamma Cuckoo Search Algorithm Validation Using Summer data........... 109 Figure 82: Current Gamma Cuckoo Search Algorithm Regression Using Summer data. ......... 110 Figure 83: New Pareto Cuckoo Search Algorithm Validation Using Autumn data. ................. 110 Figure 84: New Pareto Cuckoo Search Algorithm Regression Using Autumn data. ................ 111 Figure 85: New Pareto Cuckoo Search Algorithm Validation Using Winter data. ................... 111 xii.

(14) Figure 86: New Pareto Cuckoo Search Algorithm Validation Using Winter data. ................... 112 Figure 87: New Pareto Cuckoo Search Algorithm Validation Using Spring data..................... 112 Figure 88: New Pareto Cuckoo Search Algorithm Regression Using Spring data. ................... 113 Figure 89: New Pareto Cuckoo Search Algorithm Validation Using Summer data. ................. 113 Figure 90: New Pareto Cuckoo Search Algorithm Regression Using Summer data. ................ 114 Figure 91: Current CSLD Best Validation Performance Using Summer data. ........................ 117 Figure 92: Current CSLD Regression Using Summer data. ..................................................... 117 Figure 93: Current CSLD Best Validation Performance Using Autumn data. .......................... 118 Figure 94: Current CSLD Regression Using Autumn data........................................................ 118 Figure 95: Current CSLD Best Validation Performance Using Winter data. ............................ 119 Figure 96: Current CSLD Regression Using Winter data.......................................................... 119 Figure 97: Current CSLD Best Validation Performance Using Spring data. ............................ 120 Figure 98: Current CSLD Regression Using Spring data. ........................................................ 120 Figure 99: New CSLI Best Validation Performance Using Summer data. ................................ 121 Figure 100: New CSLI Best Validation Performance Using Summer data. ............................. 121 Figure 101: New CSLI Best Validation Performance Using Autumn data. .............................. 122 Figure 102: New CSLI Regression Using Autumn data. ........................................................... 122 Figure 103: New CSLI Best Validation Performance Using Winter data. ................................ 123 Figure 104: New CSLI Regression Using Winter data. ............................................................. 123 Figure 105: New CSLI Best Validation Performance Using Spring data.................................. 124 Figure 106: New CSLI Regression Using Spring data. ............................................................. 124 Figure 107: New CSPI Best Validation Performance Using Summer data. .............................. 125 Figure 108: New CSPI Regression Using Summer data............................................................ 125 Figure 109: New CSPI Best Validation Performance Using Autumn data. .............................. 126 Figure 110: New CSPI Regression Using Autumn data. ........................................................... 126 Figure 111: New CSPI Best Validation Performance Using Winter data. ................................ 127 Figure 112: New CSPI Regression Using Winter data. ............................................................. 127 Figure 113: New CSPI Best Validation Performance Using Spring data. ................................. 128 Figure 114: New CSPI Regression Using Spring data. ............................................................. 128 xiii.

(15) Figure 115: New CSEI Best Validation Performance Using Summer data. .............................. 129 Figure 116: New CSEI Regression Using Summer data. .......................................................... 129 Figure 117: New CSEI Best Validation Performance Using Autumn data. ............................. 130 Figure 118: New CSEI Regression Using Autumn data. ........................................................... 130 Figure 119: New CSEI Best Validation Performance Using Winter data. ................................ 131 Figure 120: New CSEI Best Regression Using Winter data. ..................................................... 131 Figure 121: New CSEI Best Validation Performance Using Spring data.................................. 132 Figure 122: New CSEI Regression Using Spring data. ............................................................. 132 Figure 123: Actual hourly power on the 19th February 2015..................................................... 134. xiv.

(16) TABLE OF CONTENTS CHAPTER 1: INTRODUCTION ................................................................................................... 1 1.1 INTRODUCTION................................................................................................................. 1 1.2 RESEARCH PROBLEM AND HYPOTHESES .................................................................. 4 1.3 RESEARCH SCOPE AND OBJECTIVES .......................................................................... 6 1.5 RESEARCH ORIGINALITY ............................................................................................... 6 1.6 PUBLICATIONS .................................................................................................................. 7 1.7 THESIS ORGANISATION .................................................................................................. 7 1.8 CONCLUSION ..................................................................................................................... 9 CHAPTER 2: ELECTRICITY SHORT-TERM FORECASTING METHODS ........................ 10 2.1 INTRODUCTION............................................................................................................... 10 2.2 SHORT TERM LOAD FORECASTING METHODS....................................................... 10 2.2.1 Neural Networks ........................................................................................................... 10 2.2.2 Fuzzy-Logic .................................................................................................................. 12 2.2.3 Neuro-Fuzzy ................................................................................................................. 13 2.2.4 Time Series ................................................................................................................... 14 2.2.5 Support Vector Machines ............................................................................................. 16 2.2.6 Regression .................................................................................................................... 16 2.2.7 Miscellaneous Methods ................................................................................................ 18 2.3 NEURAL NETWORKS ..................................................................................................... 22 2.3.1 Neural Networks Attribute ........................................................................................... 22 2.3.2 Neural Networks Components...................................................................................... 22 2.3.3. Learning Algorithms.................................................................................................... 25 2.3.4 Hidden Layer ................................................................................................................ 30 2.4 CUCKOO SEARCH ALGORITHM .................................................................................. 32 2.4.1 Original Cuckoo Search................................................................................................ 32 xv.

(17) 2.4.2 General Cuckoo Search Improvement .......................................................................... 36 2.4.3 Cuckoo Search Improvement by Distribution Functions ............................................. 38 2.5 IMPROVED BACK PROPAGATION BY CUCKOO SEARCH ..................................... 39 2.6 CONCLUSION ................................................................................................................... 40 CHAPTER 3: CURRENT CUCKOO SEARCH ALGORITHMS............................................... 42 3.1 INTRODUCTION............................................................................................................... 42 3.2 METHODOLOGY .............................................................................................................. 42 3.2.1 Fair Comparison ........................................................................................................... 42 3.2.2 Benchmarking Measure ................................................................................................ 43 3.2.3 Characteristics of Test Function ................................................................................... 43 3.2.4 Confidence Interval ...................................................................................................... 46 3.3 CURRENT PROBABILITY-BASED CUCKOO SEARCH ALGORITHMS .................. 47 3.3.1 Simulation Setup........................................................................................................... 47 3.3.2 Results and Discussions................................................................................................ 47 3.4 CURRENT DYNAMIC CHANGING CUCKOO SEARCH ALGORITHMS .................. 54 3.4.1 Simulation Setup........................................................................................................... 54 3.4.2 Results and Discussion ................................................................................................. 55 3.5 CONCLUSION ................................................................................................................... 61 CHAPTER 4: PROPOSED CUCKOO SEARCH ALGORITHMS ............................................. 63 4.1 INTRODUCTION............................................................................................................... 63 4.2 PARETO BASED CUCKOO SEARCH ............................................................................ 63 4.2.1 Pareto Distribution and Cumulative Distribution Functions ........................................ 63 4.2.2 Pareto based Cuckoo Search......................................................................................... 64 4.2.3 Probability Distribution Cuckoo Search Simulation Setup .......................................... 66 4.2.4 Probability Distribution Cuckoo Search Results and Discussions ............................... 66 4.3 DYNAMIC CHANGING SWITCHING PARAMETER CUCKOO SEARCH ................ 73 4.3.1 Cuckoo Search with Dynamically Increasing Switching Parameters .......................... 73 4.3.2 Dynamic Increasing Cuckoo Search Simulation Setup ................................................ 74 xvi.

(18) 4.3.3 Dynamic Increasing Switching Parameter Results and Discussions ............................ 75 4.4 CONCLUSION ................................................................................................................... 83 CHAPTER 5: APPLICATION OF CUCKOO SEARCH ALGORITHMS TO ELECTRICITY SHORT TERM LOAD FORECASTING ..................................................................................... 84 5.1 INTRODUCTION............................................................................................................... 84 5.2 IMPACT OF METEOROLOGICAL FACTORS ON SHORT TERM ELECTRICITY LOAD FORECASTING ........................................................................................................... 84 5.2.1 Temperature .................................................................................................................. 85 5.2.2 Humidity ....................................................................................................................... 85 5.2.3 Wind Speed................................................................................................................... 86 5.2.4 Wind Direction ............................................................................................................. 86 5.3 METEOROLOGICAL FACTORS AND ELECTRICITY CORRELATION COEFFICIENTS ....................................................................................................................... 86 5.3.1 South African Seasons .................................................................................................. 86 5.3.2 Correlation Coefficient ................................................................................................. 87 5.4 STLF SIMULATIONS ....................................................................................................... 88 5.4.1 Methodology................................................................................................................. 88 5.4.2 Simulations Setup ......................................................................................................... 89 5.5 RESULTS AND DISCUSSIONS ....................................................................................... 91 5.5.1 STLF using different probability distribution-based Cuckoo-search algorithms ......... 91 5.5.2 STLF using dynamic switching parameter Cuckoo-search algorithms ...................... 114 5.5.3 Next Hour Electricity Load Forecasting for 19th February 2015. .............................. 133 5.6 CONCLUSION ................................................................................................................. 136 CHAPTER 6: CONCLUSIONS AND FUTURE WORK .......................................................... 137 6.1 INTRODUCTION............................................................................................................. 137 6.2 SUMMARY ...................................................................................................................... 137 6.3 FUTURE WORK .............................................................................................................. 139 6.4 CONCLUSION ................................................................................................................. 139 APPENDIX A: MATLAB Cuckoo Search Algorithm ............................................................... 140 xvii.

(19) APPENDIX B: Copy of CityPower letter ................................................................................... 142 APPENDIX C: South African Weather Services letter ............................................................. 143 APPENDIX D: CSBP MATLAB code...................................................................................... 145 BIBLIOGRPAPHY..................................................................................................................... 148. xviii.

(20) CHAPTER 1: INTRODUCTION 1.1 INTRODUCTION Optimisation is a systematic process of determining the optimal solution to a given problem [1]. This process either solves for a minimum or a maximum value of a problem, known as an objective function or cost function. Optimisation problems can be classified into constrained or unconstrained problems. Constrained optimisation problems apply when solution/s being solved are constrained to a subset of all practical solutions, while unconstrained optimisation problems solutions span across all viable solutions set [2], [3]. Optimisation plays a significant role in solving different engineering problems including but not limited to systems design, electricity network operation, electricity generation, wireless communications routing and minimisation of energy losses during electricity transmission. Proper validations of optimisation algorithms require assessment of computational time and convergence rate in addition to the accuracy to determine the minimum or maximum values [4], [5], [6], [7] [8], [9], [10]. Some researchers have innovated optimisation algorithms based on their observations of some natural behaviours like animals, these algorithms are known as nature-inspired algorithms. In [11] a Bat-inspired algorithm was developed based on echolocation to sense distance between a bat and its surroundings. It is the first nature-inspired algorithm to use frequency tuning (the process of changing hearing frequency) so that they can identify objects around them accurately. Particle Swarm Optimisation (PSO) was innovated after observing the behaviour of animals that do not have leaders like fish and birds schooling while searching for food (potential solution) [12]. Each animal member searches for food randomly and communicates to the rest of the group when it’s close to the food source. Processing of communication from all other group members is used to determine the best food source (solution). Differential Evolution (DE) algorithm was created by Storm and Prince [13], based on population vectors. The vectors are chosen according to uniform probability distribution and the size of the population does not change during the search process. The population growth is based on mutation (generation new vector), crossover (increasing the diversity of new vectors) and selection (decision to determine if the new member not be part of the population or not) operations. DE is used for continuous space and efficient and robust [14]. Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 1.

(21) Ant and Bee algorithms were developed based on the foraging behaviour of ants and bees that use pheromone as a chemical messenger and the concentration of pheromone is regarded as an indication of quality solutions to the problem being solved [15]. Simulated Annealing (SA) is probabilistic nature-inspired algorithm often used in discrete global optimisation. It is preferred when determining an approximation global solution is important than the find an actual global solution due to limited time constraints [16]. Genetic Algorithm (GA) is about the process of generating offspring from parent solutions using three key operations. Crossover operation helps to exploit and enhance convergence to correct solution. Mutation deals with exploitation and less exploration. Selection guarantees that the correct solution remains part of the population [17]. The GA can converge easily to global optimality. Cuckoo Search (CS) algorithm is also a nature-inspired algorithm, based on brood reproductive strategy of Cuckoo birds to increase their population [17]. However, CS is more effective than other nature-inspired algorithms. In fact, DE, SA and PSO are exceptional cases of CS algorithm, hence it is not a surprise why CS algorithm outperforms them [17]. In [18] CS algorithm outperformed DE algorithm in terms of convergence speed to reach the optimum solution. In addition, CS algorithm was reported as being more computationally efficient than the PSO [19]. Another interesting nature-inspired algorithm is Firefly algorithm (FA), based on unique short and rhythmic light flashes produced by fireflies for three reasons. Firstly, the light is used as a form of communication to attract mating partners. Secondly, the flashes are used to attract potential prey and lastly, the flashes are used to remind and repel potential predators since the fireflies have bitter taste [20]. Yang [17] further confirmed that CS algorithm has two distinct advantages over other natureinspired algorithms like SA and GA; its efficient random walks and balanced between local and global searching. The CS algorithm has been used to solve optimisation problems across different industries. Baskan [21] used CS algorithm to minimise traffic congestion by improving the performance of transportation road networks. The objective function was defined as a total of travel time and invested the cost of 16 link capacity expansions. Cuckoo Search produced best results when Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 2.

(22) compared to other methods found in the literature. In [22] CS algorithm was used to maintain fault level and the voltage fluctuations within an acceptable level, thus minimise real power losses in a smart grid. Since its invention, electrical energy and power have played major roles in providing energy to many industries around the world. Electrical energy can be produced from Wind, hydro plants, coal and nuclear sources. Once challenge facing electricity utilities is to have an efficient way of storing this energy such that it is always sufficient when needed by consumers. One approach to handle this challenge is to forecast electricity demand at any given time and produce only the amount of electricity needed. Short-term load forecasting (STLF) is a process of forecasting electricity for one hour to one-week periods, and it is needed for power planning and power network maintenance [23]. The forecasting provides accurate information required by power utilities to ensure that correct power is generated, transmitted and distributed to consumers at the right time. The load forecasting is important for many reasons. It helps power utilities to plan the operations and maintain power system more efficient. In addition, it minimises the operational risk and enables the power utility to make economic and technical decision in terms of infrastructure investments. Moreover, it empowers power utility to secure required resources like manpower and fuel to produce and supply power to consumers. Furthermore, it can help power utility to manage its budgeting and cash flow more accurately [24], [25]. It is important to understand factors that affect load forecasting especially short-term load forecasting, Singh et al [26]. These factors include but not limited to the following. Time factor (factors affected by various times of the day) [27], load demand is not uniform throughout a day, for example, in the morning the load demand can be high due to geysers being switched on and some consumers prepare breakfast. During the days, the electricity demand is lower than in the morning or evening because people are at work. Weather factors like wind speed and direction, rain, cloud cover, temperature and humidity in the air can influence usage of electrical appliances like heaters and air conditions. [28], [29]. Singh et al [26] argued that price factor has a direct relationship to electricity load. This might be true for price-sensitive consumers who might start using less electricity to due to high electricity price.. Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 3.

(23) 1.2 RESEARCH PROBLEM AND HYPOTHESES A constant supply of electricity has a direct impact on economic productivity and boost investor confidence in any country, South Africa included. In the period from 2008 until 2015, South Africa experienced unprecedented power shortage and power utility was forced to implement load shedding. In the study by Katelhodt and Wacke [30], 62.8% of businesses confirmed that they heavily depend on stable and secure electricity supply. Moreover, in the same study, 44.9% of businesses in Cape Town agreed that they were severely financially affected by the energy crisis. This shortage of electricity negatively resulted in South African Reserve Bank and Eskom (power utility responsible of 95% of South African electricity generation) to revised GDP growth from 2.5% to 2.2% [31]. Maasdam [32], estimated that direct cost due to South African load shedding between 2007 and 2008 was USD 6.6 bn. The mining sector was affected most, and it contracted by 22.1%. For the South African economy to stay on a growth path and be competitive with other countries, it is important to use more accurate forecasting models with efficient global optimisation algorithm. That will result in enough electricity being generated, transmitted and distributed to all consumers. To ensure that enough electricity is generated in South Africa, it is essential that correct electricity load demand is forecasted accurately. In this thesis, improved Cuckoo Search algorithms will be developed and tested on mathematical test functions and on electricity load forecasting. Then the results will be compared current Cuckoo Search algorithms.. Three research hypotheses have been identified and stated as follows. Hypothesis I Null Hypothesis: 𝑯𝟎 Pareto based Cuckoo Search will not outperform the Lévy, Gaussian, Gamma and Cauchy based Cuckoo Search algorithms in determining global minima for benchmark test functions.. Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 4.

(24) Alternative Hypothesis: 𝑯𝟏 Cuckoo Search whose step size is derived from Pareto probability distribution will outperform other Cuckoo Search algorithms whose step sizes are based Lévy, Gaussian, Gamma and Cauchy probability distributions when used to find global minima of benchmark test functions.. Hypothesis II Null Hypothesis: 𝑯𝟎 Cuckoo Search algorithm with dynamic increasing switching parameter between local and global random walks will not outperform Cuckoo Search with constant switching parameter when tested on benchmark test functions. Alternative Hypothesis: 𝑯𝟏 Cuckoo Search algorithms with dynamic increasing switching parameter between local and global random walks will outperform Cuckoo Search with constant switching parameter when tested on benchmark test functions.. Hypothesis III Null Hypothesis: 𝑯𝟎 Cuckoo Search algorithm whose step sizes are derived from Pareto probability distribution will not outperform the other Cuckoo Search algorithms whose step sizes are derived from other probability distributions when used for short-term electricity load forecasting. Alternative Hypothesis: 𝑯𝟏 Cuckoo Search algorithm whose step sizes are derived from Pareto probability distribution will outperform the other Cuckoo Search algorithms whose step sizes are derived from other probability distributions when used for short-term electricity load forecasting.. Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 5.

(25) 1.3 RESEARCH SCOPE AND OBJECTIVES Many techniques and algorithms have been used for short-term electricity load forecasting. The nature of electricity load forecasting depends on many factors and it is for that reason that no technique or algorithm can be applied to cater for all factors involved. Short-term electricity load forecasting depends on many variables like economic factors, weather factors, price factors and random disturbances (sudden shutdown, the start of industries and widespread strikes) [26]. However, the scope of this thesis will evaluate impact of only weather factors on electricity load demand. The other factors like electricity pricing factors will not be used to determine the electricity load demand since they are generally complex and include the price for generation, the price for transmission and price for distribution. Moreover, the electricity prices are adjusted only once a year [33], and might have an insignificant impact on next hour electricity load forecasting. The economic factors and random disturbances could not be obtained. The objectives of the thesis are to accomplish the following: 1. To develop a Cuckoo Search algorithm whose random walk step sizes are derived from Pareto distribution function rather than from Lévy flight (Lévy distribution). 2. To develop Cuckoo Search algorithms with dynamically changing switching parameters between local and global random walks. 3. To compare the convergence speed of the Pareto based Cuckoo Search to those of Lévy, Gaussian, Gamma and Cauchy based Cuckoo Search algorithms on the mathematical test functions. 4. To design Artificial Neural Network that uses Cuckoo Search and back propagation as learning and forecasting algorithm. 5. To perform next hour short-term electricity load forecasting for Midrand (Johannesburg, South Africa) using weather parameters and previous hour electricity load data obtained.. 1.5 RESEARCH ORIGINALITY To overcome the limitations of Back-Propagation of getting trapped in the local minima and failing to identify global minima in error minimization for electricity load forecasting. Sometimes the Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 6.

(26) combinations of two algorithms have been implemented to reach accurate and fast results. Other approaches involved modifying algorithms with the hope of getting better results. 1. There are many continuous probability distributions with positive-valued random variables like Lévy distribution. However, not enough research studies have been done to assess the performance of Cuckoo Search when its random walk step sizes are derived from other probability distribution functions except a few. Based on the literature studies done, it has published the work whereby Pareto probability distribution has been used to derive the step sizes of the Cuckoo Search algorithm. 2. The Cuckoo Search algorithm switching parameter between local and global random walks is normally kept constant. In the theses, a dynamic increasing switching parameter is implemented.. 1.6 PUBLICATIONS The thesis is based on the materials that have been published in the following journals: 1. M. Mareli & B. Twala, Global Optimisation using Pareto Cuckoo Search Algorithm. International Journal of Advanced Computer Research, Vol (7), No.32 pp164-175 July 2017. ISSN(Print):2249-7277, ISSN(Online): 2277-7970, DOI: 10.19191/IJACR.2017.732006 2. M. Mareli & B. Twala, An Adaptive Cuckoo Search Algorithm for Optimisation. Applied Computing and Informatics (2017). DOI: 10.1016/j.aci.2017.09.001 3. Mahlaku Mareli & Bhekisipho Twala. Application Cuckoo Search Algorithms to South African Short-Term Electricity Load Forecasting. Open Science Journal of Electrical and Electronic Engineering. Vol. 5, No. 1, 2018, pp 1-10.. 1.7 THESIS ORGANISATION The rest of the thesis is organised into different chapters as follows: Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 7.

(27) Chapter 2 covers relevant literature review for electricity short-term load forecasting. It starts by highlighting different techniques used in short-term load forecasting. Then it introduces neural network method, its components and discusses how backpropagation learning algorithms are used in neural networks as a learning or training method. The chapter further continues in highlighting the drawbacks associated with the back-propagation method. The neural network structure and the size of its hidden layer play a critical role in forecasting accuracy; this chapter also discusses different strategies applied to determine the number of neurons in the hidden layer. In addition, chapter 2 discusses Cuckoo Search algorithm, and improvements were done to increase its efficiency as an optimisation algorithm. Chapter 2 further discusses how Cuckoo Search is used to address the back-propagation limitations presented early in the chapter.. Finally, chapter 2. concludes by highlighting the gaps identified in the Cuckoo Search improvements; the first is limited use of probability distributions to determine random walk step sizes. The second gap identified is lack of using dynamic changing switching parameter between local and global random walks. Chapter 3 focuses on the performance evaluation of current Cuckoo Search algorithms. It discusses research methodology, defines fair comparison of optimisation algorithms, benchmarking measure and characteristic test functions use to compare current methods. Simulation setup and results for current methods; Lévy, Gamma, Gaussian, and Cauchy based Cuckoo Search algorithms are presented and discussed. Chapter 3 also performs simulation on the constant, dynamic changing switching parameter between local and global random walks. In chapter 4, Pareto probability density function and its applications are discussed and then, a new Cuckoo Search algorithm whose random step sizes are derived from Pareto distribution is introduced. Simulation setup and results for Pareto based Cuckoo Search are discussed and compared with current methods results. This chapter also introduces other new types of Cuckoo Search algorithms using power increasing, exponential increasing and linear increasing switching parameters between local and global random walks. Results are presented and compared to the results obtained from current methods presented in the previous chapter 3. Chapter 5 discusses meteorological factors like temperature, humidity, wind speed and direction, that affect short-term load focusing. Then the chapter uses correlation coefficients to determine which factors are more relevant to include in short-term electricity load forecasting. The chapter Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 8.

(28) defines months for each of the four seasons then discusses the setup of the experiment and discusses STLF results. In chapter 6 conclusions are made in relation to research hypotheses, and the thesis results implications are discussed. Finally, the chapter highlights future research work.. 1.8 CONCLUSION This chapter defined the optimisation problems and the difference between constrained and unconstrained optimisation problems. Then different nature-inspired algorithms like Bat-inspired PSO, FA, SA, DE and CS algorithms were introduced and the fact that they can maintain a balance between local and global search was attributed as the main reason they are efficient. Then it was further clarified that PSO, SA and DE are exceptional cases on CS, hence CS is even more efficient than them. The use of electric power in many industries was provided and forms of electricity generation like coal, nuclear, wind, hydro were briefly highlighted. Then the challenges facing electricity power storage economically was discussed. Then the chapter elaborated why it is important to ensure that enough electrical power is generated, transmitted and distributed to consumers. The concept of electricity short-term load forecasting was introduced. Moreover, major factors that must be considered in load forecasting were also discussed. Electricity sector and its importance to the South African economy were discussed and research problem was introduced and justification for why it is worth solving. The chapter highlighted two research hypothesis which will be validated after the experimental results are obtained in the nest chapters. Research scope and six objectives were clearly presented, and the chapter discussed research originality. Finally, the chapter provided a list of publication upon which this thesis is based on. The next chapter reviews techniques used in electricity short-term load forecasting. Then it narrows the discussion to neural networks and after identifying that it is more efficient and easy to use compared to other methods.. Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 9.

(29) CHAPTER 2: ELECTRICITY SHORT-TERM FORECASTING METHODS 2.1 INTRODUCTION This chapter starts by introducing each method applied to electricity short-term load forecasting (STLF) and then reviews and discusses where and how each method was used and comments on the results obtained. The chapter then narrows to the Neural Networks (NN) method since it is popular and effective in solving complex problems including STLF. Different learning strategies are presented and original Cuckoo Search (CS) algorithm based on Lévy distribution is introduced. The chapter furthermore highlights some of the improvements implemented to make CS even more effective. Most prominent Back-Propagation (BP) learning algorithm is presented and its limitations are discussed. Furthermore, the BP improvements using Cuckoo Search algorithm are highlighted. Finally, the chapter concludes by identifying a gap in improving CS using probability Pareto distributions and using dynamically increasing the probability of discovering a foreign egg or probability of switching between local and global random walks.. 2.2 SHORT TERM LOAD FORECASTING METHODS These methods have been applied successfully for STLF in different markets and it is worth reviewing them in order to identify the best method in terms of efficiency and simplicity. 2.2.1 Neural Networks Neural Networks (NN) method is derived from modelling of human brains which comprises many interconnected neurons. The weights between neurons are changed during learning and thus able to solve non-linear complex problems [34]. To supply high-quality electricity energy, it is important to overcome both financial and technical challenges facing electric power utilities [35], [36], [37]. An integrated neural network (NN) has been encouraged for STLF especially when used with temperature values. Furthermore, temperature profiles improved the NN results for STLF by 60% [38], the heat index also improves Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 10.

(30) load forecasting results [39]. Moreover, NN possesses fast learning capability [40], and some scholars consider NN better than traditional regression-based methods [41]. However, statistical methods can complement NN methods in STLF like in the case of forecasting 1 to 24 hours ahead load demand [42]. Marin et al. [43] described a global NN-based model for STLF 24 hourly values for the next day for years 1989 to 1999, Spanish data confirmed that the model is more accurate than statistical methods. On the contrary, too many temperature sources like in North America can impose some challenges with respect to which values to use and how since the country is large with different temperatures. Fan et al. [44] implemented a way of combining the weather information using adaptive coefficients and obtained 25% hourly forecasting better than normal NN. Similarly, De Felice and Yao [45] introduced ensembles NN and used it for load forecasting in Rome, Italy. One issue with temperature values for STLF is the forecasting errors that occur when the season's change and load demands are less predictable. These errors can be reduced by implementing a moving data window algorithm to train multistage NN system. In [46], this forecasting method achieved hour-ahead and day-ahead forecasts within 2% and 5%, respectively. Occasionally, input data to STLF model is not always clean consequently, data pre-processing is required. One of the most promoted NN pre-processing techniques is the wavelet transformation. It decomposes data into high and low frequencies [47], [48], [49], [50], [51], [52]. In [53], Deterministic Annealing (DA) improved the NN STLF by 0.254% in forecasted average and 1.249% improvement of the maximum error. The reason for DA effectiveness is due to its aim to evaluate data classification in a sense of global organisation. Other successful pre-processing techniques involve genetic algorithm (GA) with 5% reduction in NN forecasting errors and the simulated annealing (SA) [54]. In the same manner, the learning process of NN also depends on the number of hidden layers [55]. The modified general neural networks (GRNN) algorithm can reduce the number of NN inputs. The GRNN method was tested using part of 2007 and 2008 part of nine New Zealand electrical substation loads. Despite minor errors in forecasting daily peak values, the method was fast and able to work in real-time operation [56].. Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 11.

(31) Similarly, the accuracy and computational speed of NN in STLF depends on the learning algorithm used. In [57] the back-propagation (BP) tendency of being trapped in a local minimum point (solution) was managed by applying genetic algorithm-based BP NN on 2010 to 2011 Fuzhou data. The particle swarm optimisation (PSO) used to train NN and yield the mean absolute percentage error (MAPE) of 1.9882% compared to BP-based NN with MAPE of 3.5164% [58]. GA-based NN produced training and forecasting errors of 9.51% and 13.18%, respectively. While the traditional NN produced higher traditional and forecasting errors of 12.12% and 14.31%, respectively [59]. Another GA based training algorithm was used in Mazandaran area in Iran for hourly load forecasting, maximum MAPE of 9.36%, more efficient than normal NN implementation with maximum MAPE of 10.19% was produced [60]. The echo state networks and principal component analysis based NN could forecast 10 minutes and 1 day ahead for Rome power grid [61]. It is important to realise that many studies have been dedicated to improving the training algorithms for NN, the technique of summing products of input variables with connecting weights. Firstly, Amjady [62] introduced forecast aided state estimator to train NN and forecast loads for holidays in Iran and again he and other scholars applied evolution algorithm to NN for handling STLF for the microgrids [63]. Secondly, the Bayesian and SVM learning methods for NN training results were slightly better than the NN STLF [64]. Thirdly, extreme learning machines algorithm reached accuracy with MAPE of 1.82% while BP NN obtained MAPE of 2.93% and the training computational time was reduced from 28.7seconds to 1.06 seconds in the Australian National Electricity Market [65]. Despite this substantial number of publications promoting the use of NN for STLF, Hippert et al. [66] are not convinced that NN outperforms other forecasting models. 2.2.2 Fuzzy-Logic The fuzzy-logic method maps a set of input to a set of output using several if-then rules non-linear mapping is accomplished by using membership functions like Triangular, Trapezoidal, Gaussian, Generalized bell [67].. Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 12.

(32) Fuzzy-logic (FL) is one of the effective methods applied to STLF. The linear fuzzy model based on Kalman filter and fuzzy rules has been used to estimate the model fuzzy coefficients [68]. This model produced MAPE and absolute percent error standard deviation of 0.7% and 0.84%, respectively for 24 hours forecasting. For a day-ahead load forecasting, fuzzy inductive reasoning (FIR) is used to lean past and future relation form load and weather information [69]. An evolution algorithm based on the simulated renouncing algorithm is used to select inputs for FIR model. Then the inputs variables are updated each time a new load pattern happens or when absolute errors are unacceptable (above predefined value). This method outperformed NN, regression and adaptive neuro-fuzzy inference when applied to Ecuadorian power system data. Some of the STLF errors are due to the unpredictability of electricity user behaviours for holidays and special days. To overcome this challenge, Song et al. [70] presented the fuzzy linear regression which yielded maximum percentage error of 3.75% when applied to 1996 and 1997 holidays load forecasting. In addition, the errors in STLF emanating from uncertainty and unpredictability of external factors in the electric grid are overcome by Interval Type-2 FL [71]. This model applied genetic algorithm with mean squared error, load, weather and calendar data as inputs and the model was effective in STLF. Also, FL was used to address the limitation of NN in handling forecasting weather seasonal changes. The fuzzy rules were constructed based on expert knowledge for similar day-loads [72]. The forecasting load errors were reduced by a remarkable 23% compared to NN when the method was tested on Okinawa Electric Power data in Japan. 2.2.3 Neuro-Fuzzy Neuro-Fuzzy is a hybrid of Fuzzy and Neural Networks methods, which facilitate the finding of membership functions and identifying appropriate rules. This method makes use of Neural Networks to develop fine-tuning of the Fuzzy system [73]. The inclusion of genetic algorithm to neuro-fuzzy (NF) in STLF seems to be a good combination. Firstly, the hybrid was used for STLF in the price-sensitive environment [74]. Secondly, it was applied to STLF using different day types and weather conditions and resulted in an average error. Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 13.

(33) of 2% [75]. Thirdly, when applied to the Taiwan Power Company load data, the hybrid yield MAPE of 1.269% when NN yield MAPE of 1.825% [76]. The application of NN models has been acknowledged for its accuracy in nonlinear load forecasting exercises. However, the forecasting accuracy depends on the structure of the NN model, which cannot be precisely determined without valuable experience. Mao et al. [77] implemented a combination of self-organizing fuzzy NN learning algorithm with a bi-level optimisation algorithm. NF model is also effective in STLF for special days moreover; it can also forecast both maximum and minimum loads for special days [78]. Yun [79] presented a combination of radial basis function (RBF) NN and adaptive neural fuzzy inference (ANFIS) for STLF. This hybrid makes use of a number nonlinear approach capacity of the RBF network to forecast the load on the prediction day with no account of the factor of electricity price, and then, based on the recent changes of the real-time price. It uses the ANFIS system to adjust the results of load forecasting obtained by RBF network. Hanmandlu and Chauhan [80] proposed the hybrids of wavelet fuzzy neural networks (WFNN) and fuzzy neural networks (FNN) for STLF. These hybrids were used for load forecasting for Indian Utility data and it was concluded that WFNN was superior compared to other models that were tested. 2.2.4 Time Series Time series technique involves statistical analysis of data collected at regular time intervals, used mainly for forecasting of values based on the historical values. This technique has many applications in different areas including electricity load forecasting [81]. The contribution of different time series types in STLF cannot be taken lightly, many studies have proved its effectiveness in electricity modelling. Firstly, the seasonal time series analysis in Belgian National Grid Operator where customers were clustered into eight profiles [82]. Secondly, Amjady [83] argued that most load forecasting models fail to consider daily peak load and he introduced time series model to cater for that. He got remarkable results when forecasting hourly loads and daily peak loads. Thirdly, time series outperformed NN by 10% when forecasting hourly average electricity load [84]. Fourthly, time series can project electricity behaviour for any special Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 14.

(34) day [85]. The method produced MAPE of 3.23% and 2.44% for individual household’s load and aggregation load for 30 minutes ahead. Moreover, Time series was successfully used for forecasting household-level electricity demand [86] in Ireland to confirm that individual intra-day load curves shape is predominantly affected by the electricity consumption behaviour. Again, it was used to model the daily load data from Cyprus and the results demonstrated that functional similar shape predicator outperformed time-series wavelet-kernel and NN methods [87]. Similarly, the seasonal Holt-Winter-Taylor (HWT) exponential smoothing was successfully used in the evaluation of large load data from ten European countries, six countries were tested on the half-hour and four countries were tested on hourly forecasting [88]. In addition, the HWT method was tested on nine-year data for half-hour and day ahead load forecasting and it yielded accurate results compared to NN method [89] Furthermore, Taylor [90] considered exponentially weighted methods for British and French day-ahead STLF. The best method in terms of performance was exponential smoothing formulation method. The hourly time series based STLF has been achieved by many types of time series, that is Kalman estimators in Western North America [91]; the autoregressive moving average with exogenous variable in Taiwan [92]; the nonlinear auto-regression with exogenous (NARX) model in Belgian Transmission system that resulted with mean square error (MSE) of below 3% [25] The cumulant and bispectrum-based autoregressive-moving average (ARMA) method [93] resulted in MAPE of 1.62%, while normal ARMA MAPE of 1.67% and ANN MAPE of 2.20%. For maximum errors, proposed method resulted in MAPE of 4.67%, while normal ARMA MAPE of 5.50% and NN MAPE of 6.21%. The STLF is very important in China due to fast-growing electricity loads. Wang et al. [94], presented time series-based hybrid for more accurate forecasting. Their results proved to be better than the results obtained from using support vector machine method. The increasing involvement of renewable energy sources poses challenges in STLF. Bracale et al. [95] concluded that accurate load forecasting depends on refined time series models using in Bayesian approach and the appropriate selection of data used for forecasting.. Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 15.

(35) 2.2.5 Support Vector Machines Support Vector Machines (SVM) technique applies training data (x1, y1), (x2, y2), …, (xn, yn), where xi are input vectors and yi are the associated output value of xi. SVM is a supervised learning with pre-defined training error established. For scenarios where training error is not predefined, large training data will be required for the technique to results in small error [96]. STLF can be achieved by applying a two-stage hybrid approach; the first stage involves selforganising maps network applied to cluster the input data set into several subsets. Then in the second stage, groups of 24 SVM for the next day's load profile are used to fit the training data of each subset [97]. The hybrid has three main advantages; firstly, it can tackle nonstationary in the electricity load time series. Secondly, it can distinguish regular from anomalous days and treat each day with the relevant scheme. Lastly, it can easily be improved for different power systems or segments. The effectiveness and efficiency of this hybrid in learning and accurate forecasting was confirmed after applying New York City data to forecast next day load profile Until now all STLF models have discussed small areas with same weather conditions. This is an issue for large geographical areas experiencing different weather and load demands at the same time and it is not easy to subdivide the region into sub-regions purely based on weather and load criteria. Fan et al. [98] developed a multi-region STLF model for a large region of Midwest in the United States. The model involved investigating and quantifying the weather and load variety for the region and then developed a multi-region load forecasting system using SVR. The region was divided into 24 sub-regions and day-ahead load forecasting was performed. The MAPE of 3.16% was obtained, better than the MAPE of 3.36% obtained for the entire region without subdivision. In the exercise limiting the operator involvement in the load forecasting procedure, Ceperic et al. [99] presented two improvements for SVR based load forecasting. These improvements are in model input generation and application of feature selection algorithms. The improved SVR technique yield accuracy of 20% to 23.4% when compared to New England dataset while the accuracy was for 2.5% to 34.2% on North American data. 2.2.6 Regression Regression is modelling of the relationship between depended and one or more in depended variables [100]. Liang and Cheng [101] introduced a hybrid model made from multi-regression Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 16.

(36) and fuzzy inference methods. The preliminary load forecasts are determined by multi-regression while the load correction inferences from historical information and past forecasted load errors are handled by Fuzzy Inference. The proposed model proved to be effective when used for 24 hours load forecasting on data from Taiwan Power Company. The model could find maximum and minimum STLF. The hybrid clustering algorithm involved grouping of similar characteristics data and using linear regression to determine underlying characteristics for each cluster [102]. This technique was applied to California and New York state system data and the California data was grouped into 12 clusters while New York data were grouped into 8 clusters. Compared to NN, the one-ahead load forecasting error was reduced by 7.5% and 9.8% for California and New York, respectively. The issue of STLF in Electric Power Utility of Serbia has been addressed by applying regressionbased adaptive weather sensitive algorithm [103]. This model is implemented in two steps. The first step deals with forecasting of total daily energy and the second step deals with forecasting the hourly load. The input parameters (weather and load conditions) are similar conditions of the day being forecasted obtained from the database. The 24-hour forecasts were found to be in-line with the results from other published papers. Borges et al. [104] argued that the combined aggregation methods are suitable for STLF for smart grids. They proposed bottom-up (with or without bias correction), top-down (with or without correction) and regression aggregation methods and they were evaluated using 2 datasets. The following was concluded from the results obtained; bias correction does not improve the forecasting performance and aggregation combination model is effective in smart grids STLF. Smart energy consumption (SEC) enables equipment to respond to electrical companies demand and pricing fluctuations, Guo [105]. In the SEC, accurate STLF is determined by applying regression and continuous conditional random fields (CCRF) on the relationship between outputs. Two diagonal matrix computational techniques are used to speed up CCRF. This CCRF method managed to reduce the relative error by 50% when applied to electricity.. Department of Electrical and Electronic Engineering Science University of Johannesburg. Page 17.

No results found