Time Series

implemented a combination of self-organizing fuzzy NN learning algorithm with a bi-level optimisation algorithm. NF model is also effective in STLF for special days moreover; it can also forecast both maximum and minimum loads for special days [78].

Yun [79] presented a combination of radial basis function (RBF) NN and adaptive neural fuzzy inference (ANFIS) for STLF. This hybrid makes use of a number nonlinear approach capacity of the RBF network to forecast the load on the prediction day with no account of the factor of electricity price, and then, based on the recent changes of the real-time price. It uses the ANFIS system to adjust the results of load forecasting obtained by RBF network.

Hanmandlu and Chauhan [80] proposed the hybrids of wavelet fuzzy neural networks (WFNN) and fuzzy neural networks (FNN) for STLF. These hybrids were used for load forecasting for Indian Utility data and it was concluded that WFNN was superior compared to other models that were tested.

2.2.4 Time Series

Time series technique involves statistical analysis of data collected at regular time intervals, used mainly for forecasting of values based on the historical values. This technique has many applications in different areas including electricity load forecasting [81].

The contribution of different time series types in STLF cannot be taken lightly, many studies have proved its effectiveness in electricity modelling. Firstly, the seasonal time series analysis in Belgian National Grid Operator where customers were clustered into eight profiles [82]. Secondly, Amjady [83] argued that most load forecasting models fail to consider daily peak load and he introduced time series model to cater for that. He got remarkable results when forecasting hourly loads and daily peak loads. Thirdly, time series outperformed NN by 10% when forecasting hourly average electricity load [84]. Fourthly, time series can project electricity behaviour for any special

day [85]. The method produced MAPE of 3.23% and 2.44% for individual household’s load and aggregation load for 30 minutes ahead. Moreover, Time series was successfully used for forecasting household-level electricity demand [86] in Ireland to confirm that individual intra-day load curves shape is predominantly affected by the electricity consumption behaviour. Again, it was used to model the daily load data from Cyprus and the results demonstrated that functional similar shape predicator outperformed time-series wavelet-kernel and NN methods [87].

Similarly, the seasonal Holt-Winter-Taylor (HWT) exponential smoothing was successfully used in the evaluation of large load data from ten European countries, six countries were tested on the half-hour and four countries were tested on hourly forecasting [88]. In addition, the HWT method was tested on nine-year data for half-hour and day ahead load forecasting and it yielded accurate results compared to NN method [89] Furthermore, Taylor [90] considered exponentially weighted methods for British and French day-ahead STLF. The best method in terms of performance was exponential smoothing formulation method.

The hourly time series based STLF has been achieved by many types of time series, that is Kalman estimators in Western North America [91]; the autoregressive moving average with exogenous variable in Taiwan [92]; the nonlinear auto-regression with exogenous (NARX) model in Belgian Transmission system that resulted with mean square error (MSE) of below 3% [25] The cumulant and bispectrum-based autoregressive-moving average (ARMA) method [93] resulted in MAPE of 1.62%, while normal ARMA MAPE of 1.67% and ANN MAPE of 2.20%. For maximum errors, proposed method resulted in MAPE of 4.67%, while normal ARMA MAPE of 5.50% and NN MAPE of 6.21%.

The STLF is very important in China due to fast-growing electricity loads. Wang et al. [94], presented time series-based hybrid for more accurate forecasting. Their results proved to be better than the results obtained from using support vector machine method. The increasing involvement of renewable energy sources poses challenges in STLF. Bracale et al. [95] concluded that accurate load forecasting depends on refined time series models using in Bayesian approach and the appropriate selection of data used for forecasting.

2.2.5 Support Vector Machines

Support Vector Machines (SVM) technique applies training data (x1, y1), (x2, y2), …, (xn, yn), where xi are input vectors and yi are the associated output value of xi. SVM is a supervised learning with pre-defined training error established. For scenarios where training error is not predefined, large training data will be required for the technique to results in small error [96].

STLF can be achieved by applying a two-stage hybrid approach; the first stage involves self-organising maps network applied to cluster the input data set into several subsets. Then in the second stage, groups of 24 SVM for the next day's load profile are used to fit the training data of each subset [97]. The hybrid has three main advantages; firstly, it can tackle nonstationary in the electricity load time series. Secondly, it can distinguish regular from anomalous days and treat each day with the relevant scheme. Lastly, it can easily be improved for different power systems or segments. The effectiveness and efficiency of this hybrid in learning and accurate forecasting was confirmed after applying New York City data to forecast next day load profile

Until now all STLF models have discussed small areas with same weather conditions. This is an issue for large geographical areas experiencing different weather and load demands at the same time and it is not easy to subdivide the region into sub-regions purely based on weather and load criteria. Fan et al. [98] developed a multi-region STLF model for a large region of Midwest in the United States. The model involved investigating and quantifying the weather and load variety for the region and then developed a multi-region load forecasting system using SVR. The region was divided into 24 sub-regions and day-ahead load forecasting was performed. The MAPE of 3.16%

was obtained, better than the MAPE of 3.36% obtained for the entire region without subdivision.

In the exercise limiting the operator involvement in the load forecasting procedure, Ceperic et al.

[99] presented two improvements for SVR based load forecasting. These improvements are in model input generation and application of feature selection algorithms. The improved SVR technique yield accuracy of 20% to 23.4% when compared to New England dataset while the accuracy was for 2.5% to 34.2% on North American data.

2.2.6 Regression

Regression is modelling of the relationship between depended and one or more in depended variables [100]. Liang and Cheng [101] introduced a hybrid model made from multi-regression

and fuzzy inference methods. The preliminary load forecasts are determined by multi-regression while the load correction inferences from historical information and past forecasted load errors are handled by Fuzzy Inference. The proposed model proved to be effective when used for 24 hours load forecasting on data from Taiwan Power Company. The model could find maximum and minimum STLF.

The hybrid clustering algorithm involved grouping of similar characteristics data and using linear regression to determine underlying characteristics for each cluster [102]. This technique was applied to California and New York state system data and the California data was grouped into 12 clusters while New York data were grouped into 8 clusters. Compared to NN, the one-ahead load forecasting error was reduced by 7.5% and 9.8% for California and New York, respectively.

The issue of STLF in Electric Power Utility of Serbia has been addressed by applying regression-based adaptive weather sensitive algorithm [103]. This model is implemented in two steps. The first step deals with forecasting of total daily energy and the second step deals with forecasting the hourly load. The input parameters (weather and load conditions) are similar conditions of the day being forecasted obtained from the database. The 24-hour forecasts were found to be in-line with the results from other published papers.

Borges et al. [104] argued that the combined aggregation methods are suitable for STLF for smart grids. They proposed bottom-up (with or without bias correction), top-down (with or without correction) and regression aggregation methods and they were evaluated using 2 datasets. The following was concluded from the results obtained; bias correction does not improve the forecasting performance and aggregation combination model is effective in smart grids STLF.

Smart energy consumption (SEC) enables equipment to respond to electrical companies demand and pricing fluctuations, Guo [105]. In the SEC, accurate STLF is determined by applying regression and continuous conditional random fields (CCRF) on the relationship between outputs.

Two diagonal matrix computational techniques are used to speed up CCRF. This CCRF method managed to reduce the relative error by 50% when applied to electricity.

2.2.7 Miscellaneous Methods

This subsection grouped various type of STLF methods which fall outside the groups identified in the previous subsections.

The use of Monte Carlo predictions to estimate power generation costs is very interesting per Valenzuela et al. [106] who made the following conclusions; Firstly, production costs were obtained when both temperature values and hourly load were considered. Secondly, STLF both expected, and variance values of the production costs depended on their initial state of the generating units. Thirdly, they observed that short-term production costs can be made more accurate when using temperature forecasts. Fourthly, they found that load uncertainty is a significant component of the total uncertainty associated with production costs for STLF.

Simulated Annealing (SA) methods can be used to solve the unavailability of the generating units and the uncertainty of the STLF unit commitment [107]. The method proved to be capable of solving unit commitment constraints and efficient while being put to test with 26 generating units.

Elattar et al. [108] presented an STLF method using an encoding scheme of evolution algorithm.

The network structure is determined by each chromosome. This method outperformed SVR when tested using real data.

STLF can be determined from historical data. Hoverstad et al. [109], proposed a three-stage model consisting of pre-processing forecasting and post-processing. For the pre-processing stage, seasonality is extracted, and then genetic algorithm forecast in the forecasting stage. The seasonality is then added in the post-processing stage.

The increase of renewable energy especially wind power has challenged power systems management in keeping the grid reliable and secure. Livani et al. [110] investigated the use of secondary function (SF) method to predict wind and load forecasts. This method employs Markov models and Viterbi algorithm for short-term congestion and ramp predictions using power system conditions, historical data as well as wind and load forecasts. The SF method was evaluated on the Bonneville Power Administration data and the simulation results proved the superior performance, the management can use it to make reliable control decisions.

It is important to improve network operation and planning in a competitive market. Villalba and Bel [111] introduced a Hybrid Demand Model to enhance STLF. The first phase of the model deals

load estimates at certain points in the network. The second phase forecasts on nonlinear STLF using temperature and load data as inputs for NN. The advantages of this model include the reusability of the model without having to collect more data. Using real data from distribution network with load peak values of 0.5MW to 2MW, the model was applied and resulted in MAE of 7%. These results confirmed the effectiveness of the model in STLF.

Abdel-Aal [112] proposed Abductive Network model for next-hour and next-day load forecasting.

This model can automatically select influential inputs from input-output. The model was tested at five-year temperature and load data, and the sixth-year data was used for forecasting. The models produced overall MAPE of 2.67%. For the next-hour forecasting, the model produced MAPE of 1.14%. Although the model might seem to be accurate; its main disadvantage is that 24 dedicated models had to be developed to forecast next-day hourly loads.

The load forecasting is facing some challenges due to smart grid and renewable energy sources.

Goude et al. [113] proposed the application of Semi-Parametric Adductive Models (SPAM) to address these challenges in STLF. The model was tested in the France substations using data from January 2006 until December 2011 and the following observations were made; the results were good and sufficient, the impact of seasonality and electricity tariffs on the load usage could be established. Moreover, the STLF also depends on accurate temperature predictions. The challenge attributed to renewable energy source can be overcome by incorporating the dependent variables like wind speed and cloud cover duration. The other challenges due to smart grids can also be compensated by treating them like large electrical appliances.

Chakhchoukh et al. [114] proposed a load forecasting method using multivariate ratio-of medians-based estimator (RME). They used the model to forecast a day-ahead in France. The RME outperformed other methods; it is fast in execution, simple and easy to use for online implementation.

In power systems, states and noises do not obey Gaussian distributions because of unexpected distributions. For STLF, Han et al. [115] proposed the use of feedback error corrections to overcome this uncertainty. The load of 288 data sets with an interval of 5 minutes of 500KV nodes was used for testing. The first seven data sets are used to start the model build algorithm and the remaining 281 datasets used for testing.

Grid stability and effectiveness are increased by accurate STLF. However, the load fluctuations make forecasting less accurate due to various changing factors. The study by Alamaniotis et al.

[116] suggested the use of Kernel-based Gaussian Process (GP) they used historical data from 1999 to 2000 from Chicago for the 5 minutes predictions. Three GP forecasters, GP with the sum of Matern, NN and Noise kernel; PG with the sum of Gaussian and Noise kernels and GP with the sum of Rational Quadratic and Noise kernel were developed and tested. Despite the highly accurate results obtained from ensemble kernel-based GP (MAPE of 0.2236%) while ARMA resulted in MAPE of 0.4867%, they recommended that further studies be made of the genetic algorithm for vector optimisation.

Most STLF models assume that load data can always be available when it is required for forecasting. However, communication channel providing data can fail which can result in latest data not being available. Chaojun et al. [117] recommended time-forward Kriging method. This method can forecast the loads by utilizing load information from neighbouring regions. The method was used for 5 minutes’ load forecasting using New York load data and 1-hour load forecasting using load data from PJM transmission company. The average forecasting errors are 2.97% and 5.26% for New York and PJM, respectively. Even though the time-forward Kriging method being used to forecast load by using data from other regions, the forecasting error will depend on the correlation between the load being affected by communication failure and the available load. Moreover, this method assumes that not all communication channels fail at the same time.

Despite a wide range of load forecasting techniques available, it is challenging to effectively integrate numerous factors in load forecasting. Semi-parametric addictive models can be used to determine the relationships between load and various load determining factors like day of the week [118]. The Addictive model and other models were tested for half-hour electricity demand for up to seven days ahead for power systems in the Australian National Electricity Market. Addictive model with average MAPE of 1.88% outperformed both NN and Hybrid methods with MAPE averages 2.81% and 2.14%, respectively.

Aneiros et al. [119] presented two models for residual demand in spot markets. The first model is a Functional nonparametric model and it estimates the residual demand as a function of past

residual demands. The second model is a Semi-Functional Partial linear model and it uses electricity demand and wind power forecasts as descriptive variables. These models were tested using two-year data from Spain and it was concluded that the proposed models outperformed the time series models.

To understand the error associated with load forecasting, Garcia et al. [120] introduced two indicators. The first indicator is power spectral density, this indicator determines the load series power by using discrete Fourier series. The second indicator is load predictability, it was concluded that load series can be predictable if its power spectral distribution has few frequencies that carry significant power compared to the power found in the original signal.

When analysing the multiple sources of electrical energy, Zhang and his team [121] identified a potential issue of not being able to determine load forecasts in brief time horizon. They then decided to propose big data technologies for STLF for each individual load before adding different loads to determine the total load for an area. The forecasting method consists of five-modular residing in the application layer, these modules are; cluster analysis, Association analysis, Decision tree, model selection and finally forecasting module. The application layer runs on top of computation layer which sits on data management layer and which sits on data storage layer. When used for hourly load forecasts the following relative errors of a customer were obtained; the proposed method was 1.61%, SVM was 3.09% and NN was 3.40%. These results confirmed that the proposed method outperformed other methods.

Electricity is one of the most convenient forms of energy for households and industrial/commercial appliances. The introduction of smart meters has even allowed power utilities to remotely collect and analyse electricity data from consumers. Quilumba et al. [122] introduced the concept of customer clustering by assessing load consumptions similarities to perform STLF. This method was used for day-ahead load forecasting from two residential databases. The results from database 1 and database 2 confirmed that MAPE was reduced by 0.5% and 1.07%, respectively.

2.3 NEURAL NETWORKS

2.3.1 Neural Networks Attribute

Neural networks have very interesting characteristics that make them suitable for solving all kinds of engineering problems from basic to complex [123]. Firstly, neural networks are flexible and can adapt during learning process by changing the connections between the neurons. These can result in being neurons added or deleted from the network. The changes can also involve changing the connection weights between the neurons. Secondly, modelling complex problem can be time-consuming and is prone to errors, however, neural networks are able to map nonlinear inputs to an output. Thirdly, the neural networks can solve problems even if there is some missing or noisy data provided since it learns through experience. Fourthly, neural networks have high processing speed due to their parallel and distributed architecture.

2.3.2 Neural Networks Components

Neural Networks are made of several neurons connected to each other via weights. The topology of each neural network is determined by the way its neurons are arranged and direction of connection between them. Each neuron is associated with a function that determines the relationship between its inputs and output.

2.3.2.1 Neuron

A building block of neural networks is a neuron with weighted connections and bias as shown in Figure 1. Each input is multiplied by a corresponding weight and the products are sum together with a bias value. Then the result is passed through an activation function to produce final output of a neuron.

The inputs are represented by (x1, x2,….,xn), weights are represented by (w1, w2,...,wn) and the bias is represented by b. Σ is the summation sign, for adding all weighted inputs.

2.3.2.2 Propagation function

Propagation function receives output from other neurons and determined the input for the

Propagation function receives output from other neurons and determined the input for the