Machine learning is mushrooming in time series systems due to the integration of traditional statistical approaches. As described in chapter three, unlike the language model, speech recognition and computer vision where models are applied directly to the data [65, 92, 93], time series requires efficient modelling due to outliers inherited from the data.
In section 4.4, data is differenced to improve performance, the core approach of the Box- Jenkins time series model [94] which is the d component of the ARIMA model. The p and q
parameters as modelled in [95, 96] can either be generated using machine learning tools28 or
the traditional autocorrelation function (ACF) and partial autocorrelation functions (PACF) as described below.
4.8.1 Autocorrelation functions (ACF)
For wind speed and wind power of a given series, selection of the decomposition level remains one of the most important decisions. However, to the best of the author’s knowledge; there is
28Using the traditional ML technique to deduce the p, d, q components on Sklearn-metrics ARIMA model
87 | P a g e
no theoretical criterion for the selection of theoretical decomposition level. Scikitlearn packages in python results in the use of machine learning algorithm developed by Python developers to solve issues of decomposition, traditional ACF has over the years, employed decomposition to solve the seasonality of the wind series data. However, the motivation of the ACF usage as described by [97] is the ability to characterize ‘’periodicity or self-similarity’’ which invariably is the scale invariance property of time series systems (see Appendix A (i)).
Figure 4. 14: Autocorrelation plot of the wind series.
Furthermore, time series components relies on ACF to verify whether observations such as wind speed has trend and or seasonality as observed in Figure 4.14, then report the extent or sufficiency level in relation to the wind decomposition. Conversely, when the trend and seasonality component seen in Figure 4.15 of the wind speed series is similar to the ACF, we can reasonably conclude that the decomposition of moving average (MA) or auto-regression (AR) provides sufficient accuracy.
Autocorrelation values are statistically obtained by the use of [98, 99] for the associations between data in time series separable at different lags of times. In other words, time series y with Z data points at index lag of τ is estimated with Eq. (4.44) where y is the average of time series, shown in Figure 4.14.
88 | P a g e ρ(τ) = ∑Z−τz [y(m+τ)−ӱ][y(z)− ӱ]∑ [y(z)−ӱ]2 Z−τ m=1 (4.44) 4.8.2 Partial autocorrelation function (PACF).
Like the decomposition discussed above, the partial autocorrelation function is used to measures the correlation between observations of the wind series that are separated by t time units – 𝑦𝑡and 𝑦𝑡−1, after adjusting for the presence of all the other terms of shorter lag
yt−1, yt−2, , … . , yt−t−1 as shown in Eq. (4.36) of [100] gave a detailed description of PACF.
Figure 4. 15: PACF of the wind series.
Furthermore, in terms of the ARIMA model, the relationship with PACF is related to the p component of the ARIMA component. From the test, the component is as shown in Table 4.2 below.
Research over the recent years has developed algorithms and systems that can derive these ARIMA components programmatically without differencing, correlating and partially correlating the signals, this result in the machine learning aspect of component derivation.
80 70 60 50 40 30 20 10 1 1.0 0.8 0.6 0.4 0.2 0.0 -0.2 -0.4 -0.6 -0.8 -1.0 Lag Pa rti al A ut oc or re la tio n
89 | P a g e
4.8.3 Machine learning ARIMA Derivation.
The grid search technique29 is the machine learning approach to calculate the best ARIMA
(𝑝,𝑑,𝑞) components. Applying grid search hyper-parameter for the p, d, q components to the training and test sets is ideal for optimum performance due to the model building across each component.
The approach in this process is iteration through parameter combinations for each possible model combination [101]. The grid search method depends on data size and is expensive computationally, it relies on the performance of the system processor and RAM in use. In addition, grid searching can be used to tune for performance measures, like the AIC, AUC criteria and so on. The model tuning result is a measure of the RMSE statistical quantity for error search accuracy. In our case, about 90% evaluated RMSE error on each of the trials – Table 4.2 were reported meaning that the search has the best (p, d, q) components. However, the code utilised in grid searching the data is presented in Appendix a (ii).
Table 4. 2: Comparing the p, d, q component of ARIMA and ML derivation.
Sample wind data ARIMA (P,D,Q) ML-ARIMA (P,D,Q) 720 (0.1, 1.2, 2.3) (0,1,2) 1440 (2.4, 1.2, 2.6) (2,1,1) 2160 (1.3, 0.3, 1.6) (1,0,2) 2880 (3.3, 0.3, 0.4) (2,1,2) 3600 (4.3, 1.2, 3.2) (1,1,2) 4230 (0.3, 1.0, 3.2) (1,2,5) 5040 (2.3, 1.2, 0.2) (0,1,4) 5760 (3.2, 4.2, 3.2) (2,2,3) 6480 (0.3, 1.0, 2.2) (3,1,2) 7200 (2.3, 4.2, 2.2) (2,3,2)
4.8.4 Time Series Residual Component.
This is the difference between an observed value (y) and its corresponding fitted value (ŷ). For example, the scatterplot of Figure 3.12 (chapter 3) that plots men's weight against their height,
90 | P a g e
the regression line plots the fitted values of weight for each observed value of height. Suppose a man is 6 feet tall and the fitted value of his weight is 190 lbs. If his actual weight is 200, the residual is 10. If his actual weight is 175, the residual is -15. Residual values are especially useful in regression and ANOVA procedures because they indicate the extent to which a model accounts for the variation in the observed data.
91 | P a g e