Several contributions and outcomes made in this research are highlighted in this section.
• A novel optimisation algorithm, FOLA, is developed in this research, by im- proving the two main aspects of Fig. 1.2: action selection and learning al- gorithm. FOLA consists of multiple automata, where each automaton un- dertakes dimensional search on a selected dimension of the solution domain. FOLA has the ability of memorising history by estimating and updating the values of states that have been visited. With these approaches, FOLA is able
1.5 Contributions of the Research 28
to undertake search in continuous states and achieve accurate solutions effi- ciently. In contrast to EAs which adopt population-based search, FOLA re- duces the computation complexity by sequential dimensional search. Unlike EAs which mainly memorise the global optimal solution and use it to guide search, FOLA memorises the previous performance evaluations, which can provide more information for future seach and thus increase the efficiency. The experimental results have demonstrated that FOLA has better perfor- mance than a number of EAs.
• Multi-objective optimisation by learning automata (MOLA) is presented to
solve complex multi-objective optimisation problems. MOLA capitalises on the merits of the structure of multiple automata, the dimensional search, the dividing of the dimensional search domain into cells, state memories via which the search action is carried out, and the process of learning from the best solution and neighboring solutions. MOLA is able to find accurate non- dominated solutions and evenly distributed Pareto fronts, when solving the complex optimisation problems which have complicated Pareto set. The mer- its of MOLA have been demonstrated, in comparison with two latest develop-
ment of multi-objective EAs,i.e.MOEA/D and NSGA-II.
• The proposed FOLA has been applied to solve the power system dispatch
and voltage stability problem. This problem has received considerable at- tention, and is widely used in power system operation and planning, due to that power system dispatch is an important factor from the perspective of cost in modern energy systems, and the voltage instability is a major power sys- tem weakness resulting in severe detriments with economical, technical and social dimensions. This problem can be formulated as a highly constrained complex optimisation problem, which has the nature of non-differential, non- linearity and non-convex. For this problem, the applicability of FOLA has been investigated based on standard IEEE 30-bus and 57-bus power systems. Comparison has been carried out among FOLA, improved GA and PSO. The experimental results show that FOLA has superior performance over the other
two algorithms.
• With the increasing demand of power nowadays and the stress of the resources which can be used to generate power, renewable energy becomes more and more important in many countries, especially wind energy. Wind power in- terconnected to power systems brings new challenges to power system eco- nomic operation. It is imperative to study how to efficiently solve optimal power flow formulation which is integrated with wind power, so as to achieve an optimal solution to the specific power system objective functions. The application proposed here aims to achieve economic power system dispatch and voltage stability enhancement in dynamic wind power integrated systems. IEEE 30-bus, 57-bus and 118-bus power systems, which are integrated with time-varying wind power, have been employed to evaluate the performance of FOLA for the problem. The experimental results show that FOLA is able to optimise the problem efficiently.
• MOLA has been applied to solve the problem of economic emission dispatch
and voltage stability enhancement in wind power integrated power systems, due to the increasing importance of pollutant emission and wind power. The IEEE 30-bus power system and new England test power system are modi- fied to integrate fixed-speed and variable-speed wind power generators re- spectively. MOLA has been applied to solve the problem, and the simulation results have presented the superiority of MOLA over NAGA-II and MOEA/D.
• Deregulating the power market creates more competition and more trading
mechanisms for market players. However, the emergence of deregulated elec- tricity markets poses new challenges to the solution of the optimal power flow problem. MOLA is applied to maximise social benefit and enhance voltage stability in deregulated electricity market simultaneously. MOLA is compared with MOEA/D and NSGA-II, on the challenging optimisation problems on the IEEE 30-bus power system. MOLA can obtain accurate Pareto optimal solutions and find wider and evenly distributed Pareto fronts.
1.5 Contributions of the Research 30
The publications produced from this research work are listed in this section as follows:
1. Q.H. Wu and H.L. Liao. Function Optimisation by Reinforcement Learning
For Power System Dispatch and Voltage Stability. InProc. of PES General
Meeting, pages 1-8, Minneapolis, Minnesota, USA, 2010.
2. H.L. Liao and Q.H. Wu, Multi-Objective Optimisation by Reinforcement Learn-
ing. InProc. of IEEE World Congress on Computational Intelligence, pages
3374-3381, Barcelona, Spain, 2010.
3. Q.H. Wu and H.L. Liao. High-dimensional Optimisation by Reinforcement
Learning. InProc. of IEEE World Congress on Computational Intelligence,
pages 2808-2815, Barcelona, Spain, 2010.
4. H.L. Liao, Q.H. Wu, and L. Jiang, Multi-Objective Optimisation by Rein- forcement Learning for Power System Dispatch and Voltage Stability. In Proc. of PES Conference on Innovative Smart Grid Technologies Europe, pages 1-8, Gothenburg, Sweden, 2010.
5. H.L. Liao and Q.H. Wu. Optimal Power Flow in Wind Power Integrated Sys-
tems using Function Optimisation by Learning Automata. InProc. of PES
General Meeting, pages 1-8, Detroit, MI USA, 2011.
6. M.S. Li, Q.H. Wu, H.L. Liao, W.J. Tang and Y.S. Xue. Optimal Power Flow with Environmental Constraints Using Paired Bacterial Optimiser. InProc. of PES General Meeting, pages 1-8, Detroit, MI USA, 2011.
7. Z. Ji, J.R. Zhou, H.L. Liao and Q.H. Wu, A Novel Intelligent Single Particle Optimiser,Chinese Journal of Computers, 33(3):556-561, 2010.
8. Z. Ji, H.L. Liao, and Q H. Wu,Particle Swarm Optimisation and Its Applica- tions, Science Press (China), 2009.
Developments of Learning
Automata-based Optimisation
Chapter 2
Functional Optimisation by Learning
Automata
2.1
Introduction
Due to the increasingly complex real-world optimisation problems and the insuf- ficiency of classical optimisation algorithms in solving multi-modal problems, bio- logically inspired optimisation algorithms, which incorporate the behaviours of bi- ological principle into their algorithmic framework [5], have been comprehensively investigated over the last few decades. Notably EAs, which are introduced in Section 1.2.2, have been widely applied for problem solving over the last twenty years, rang- ing from scientific problems to engineering applications. However, the performance of EAs is interfered by the randomness and the large population sized applied, which cause unexpected redundant computational load, and thus reduce the efficiency of the algorithms in many applications [32]. Besides EAs, learning automata methods have also been applied to resolve optimisation problems. For instance, the method of continuous-action learning automata has been applied for stochastic optimisation [68][69]; the genetic learning automata was proposed to solve function optimisation problems [70]. In contrast to EAs, the learning automata methods are less popularly applied for solving complex function optimisation problems, despite having a solid theoretical background and having a significant impact on many areas of systems
control and pattern classification [64][66][67].
This chapter presents a novel algorithm: FOLA (Function Optimisation by Learn- ing Automata). FOLA consists of multiple automata, in which the number of au- tomata used is equal to the total number of dimensions of the solution domain. Each automaton is responsible for searching on one specified dimension, and dimensional states on this dimension are considered as the states of this automaton. Before an ac- tion is taken, the automaton, at a dimensional state, selects a path from two possible paths, according to the probability which is calculated from the path value that is es- timated to indicate the potential of finding a better solution if the automaton searches down on this path. Then an action takes place by moving on the selected path with a certain step length. To evaluate the effectiveness of the action, a reinforcement signal, in a scale value, is generated by the environment.
In order to avoid a large amount of computation caused by the huge number of states involved in the continuous search domain, each dimension is divided into a certain number of cells, and each cell is assigned with a cell value. The cell value is updated based on its past values, and the path values and reinforcement signal obtained when the dimensional states located in the cell are visited. FOLA has been found capable and efficient in finding accurate solutions to complex optimisation problems. Its merits have been demonstrated, in comparison with other EAs, by evaluating it on a number of uni-modal or multi-modal benchmark functions re- spectively. The experimental results have shown that FOLA outperforms the other EAs, in terms of accuracy and efficiency, especially for solving high-dimensional optimisation problems.