In regression problems, one of the major processes is a measure of algorithm performance when fitted to a model, in other words, training. This measure is with respect to error reduction during the training process. This implies that the increase in error reduces algorithm adaptability on data samples, which in turn reduces performance on unseen (test) data, after training. In addition, the measure is scale dependent and has the ability to compare forecasting errors of different models over a particular data sample, hence not between datasets.
3.6.1 Root Mean Square Error.
In sequence prediction, the value of the predicted model is measured by how it performs in understanding or training sample data. Research reported by [4, 18, 53, 69, 70] measures the performance of an ARIMA model over several training samples using RMSE as shown in Eq. (3.17); MSE, MAE performance measures metrics for a univariate system. In a bid to compute the performance of different algorithms – SVR, ARIMA, PAR and EDM over a sample set of 650 samples, [53] employed RMSE, MSE, nRMSE to measure the independent performance of the algorithms. On the other hand, [70] presented RMSE as the best metrics and further reported that SVR is the best algorithm that learns the behaviour and pattern of the samples.
RMSE = √∑Tt−t(Pt−yt)2
n (3.22)
Depending on datasets and problems, RMSE and MSE appear to be the most used metrics in for time series and sequence prediction. This is due to the effect on unseen data as RMSE in proportional to the size of the squared error. This means that larger variations of errors have a disproportionately large effect on the square root of the error. However, the consequence of squaring this error is the sensitivity to outliers.
51 | P a g e
In a regression problem shown in Figure 3.13 RMSE is calculated as Eq. (3.23), which means that
for a predicted value 𝑦𝑡 of a time sequence t of dependent regression variable 𝑃𝑡 observed over T
times and computed in a T-variable.
Figure 3. 12: Sample of error measurement.
In a cross-sectional sequence of data, t is described as i. While T is seen as n. RMSE is simply used to compare differences between two variables in some discipline. In an unbiased estimator, the RMSE is also described as the square root of the variance, which in turn is known as the standard deviation. RMSE is of different variants in terms of use. Other performance metrics for time series and sequence forecasting are the nRMSE, MAE, MSE and Mean absolute percentage error (MAPE) as discussed below.
3.6.2. Normalized RMSE (nRMSE)
This is applied when data is of different scales, although in the literature, research has not recorded consistent means of normalization. This means that the mean and range of RMSE becomes a common choice for normalizing root mean squared error. The equation of nRMSE shown in Eq. (3.23) is expressed as a percentage metric having a high value that indicates high residual variance. The metrics are dependent on sample size for better comparison.
nRMSE = 𝑅𝑀𝑆𝐸 (𝑦𝑚𝑎𝑥 − 𝑦𝑚𝑖𝑛) (3.23) 0 1 2 3 4 5 6 0 1 2 3 4
y
predicted actualy = mx + b
𝑥
52 | P a g e
where 𝑦𝑚𝑎𝑥and 𝑦𝑚𝑖𝑛 are the maximum value and minimum value of the sample data respectively.
3.6.3. Mean Absolute Error (MAE)
This is described as the absolute average difference between two variables having the X and Y as in Figure 3.12 fundamentally, MAE is easier to understand than RMSE due to its interpretability. From Figure 3.12, MAE shown in Eq. (3.24) is the average vertical distance between each point, in order words; the average is in the form
𝑀𝐴𝐸 = ∑𝑛𝑖=1|𝑦𝑖 − 𝑥𝑖|
𝑛 (3.24)
where 𝑦𝑖 is the predicted variable and 𝑥𝑖 is the actual variable.
3.6.4. Mean Absolute Percentage Error (MAPE)
This metric, represented by Eq. (3.25) is similar to RMSE; MAPE has the absolute value summed for every forecasted point in time divided by the number of fitted points, n. the factor multiplied by 100 made it a percentage error.
𝑀𝐴𝑃𝐸 = 100 𝑛 ∑ | 𝐴𝑡− 𝐻𝑡 𝐻𝑡 | 𝑛 𝑡=1 (3.25)
From the Eq. (3.20) above, 𝐴𝑡 represents the actual value, while 𝐻𝑡 is the predicted value. The
difference between 𝐴𝑡 and 𝐻𝑡 is however divided by actual value 𝐻𝑡. Research employing MAPE
experiences drawbacks; it suffers poor generalisations on zero values or missing values. In addition, MAPE performs poorly on predictions that cannot exceed 100% for forecasts that are too low and fails forecasts, which are too high. In order words, no upper limit to percentage error. Hence, the reasons why MAPE is not statistically accurate when compared to other metrics such as RMSE, MAE, nRMSE, etc.
53 | P a g e
3.6.5. Mean Square Error (MSE)
This metric measures the average of squares of errors and is often, addressed as a risk function, which corresponds to the expected value of the quadratic loss. The difference occurs because of randomness and is controlled by the square root term as in RMSE. Furthermore, in MSE metrics, values closer to zero are better estimators as shown in Eq. (3.26)
𝑀𝑆𝐸 = 1
𝑛∑ (𝑃𝑖− 𝑦𝑖) 2 𝑛
54 | P a g e
Chapter 4.
Wind-Farm Power Prediction Methodology
This Chapter presents various models and steps to achieve the research objective. It studies the mathematical literature underlying the principles of wind speed prediction for generated wind power outputs and its logical relationships to artificial neural network (ANN) especially for wind-farm data analysis and prediction. New models formulated where necessary. Data collection and statistical models for wind data analysis, which involves the utilization of empirical correlations and standard probability distributions to estimate the parameters necessary to develop the output wind-power operation process.
One of the major focuses of wind research is to understand the relationship between wind power statistical distributions and atmospheric variables. These variables are turbulence, wind speed (see section 3.1.2) that is dependent on a broad range of temporal and spatial scales in a geographic area as shown in Figure 4.1. The distributions on the other hand are the Weibull, Rayleigh, and the wind power density. Verification of numerical models by fieldwork and statistical analysis enhance understanding of atmospheric variables both in horizontal and vertical landmasses above earth or sea surfaces. Operators and wind developers however rely on high-resolution remote sensing computer simulation to provide useful prediction, which in turn enables them to select and operate, wind farm sites efficiently. The high resolution computer simulation are built on sophisticated models such as recurrent neural networks
(RNNs), convolutional neural networks (CNNs), hidden Markov models, (HMM) [71-73]and
so on, to understand wind complexities and atmospheric instabilities. Therefore, understanding wind dynamics by improving accuracy of wind prediction is critical to grid operators to balance
55 | P a g e
power generation by decreasing or increasing production from other sources – biomass, geothermal, hydroelectricity, natural gases (coal), etc.