Typically, when constructing a model to use for forecasting, it is common to perform some sort of preprocessing. We propose here that testing for changepoints in the historical data should form a part of this preprocessing step. This is our first approach
Time X 0 50 100 150 200 −2 0 2 4 0 5 10 15 20 0.0 0.2 0.4 0.6 0.8 1.0 Lag A CF
Figure 3.2: (a) An i.i.d. Gaussian time series Xt with change in mean and (b) its
autocovariance function (ACF).
to using changepoints in forecasting.
By way of example, consider a time series of independent identically distributed Gaus- sian observations of length 200 exhibiting a change in mean from zero to three at time 100. Figure 3.2 shows one realisation of this process together with the autocorrelation of the time series. It is evident that despite the observations being i.i.d., autocorre- lation is present. This is an example of a lower order structure change affecting the estimates of higher order structures in a time series, and has also been observed by Norwood and Killick (2018).
If we naively used the forecast package (Hyndman et al., 2007) to fit a model to this data, we would typically first difference the data and then fit a time series model to it. By analysing the autocorrelation function (ACF) and partial autocorrelation function (PACF) of the differenced data, the appropriate model is an ARIMA(0,1,1) model. Figure 3.3a shows the residual errors given by an ARIMA(0,1,1) model fit to the data. We can see generally larger residuals around the location of the change in mean.
Instead of differencing the data, one approach we can take is to detect the change in mean during a preprocessing step, and then incorporate it explicitly into our forecast- ing model as a dummy variable. When we do so, we correctly identify that there is
Time Residuals 0 50 100 150 200 −2 −1 0 1 2 3 (a) Time Residuals 0 50 100 150 200 −2 −1 0 1 2 3 (b)
Figure 3.3: The residuals for Yt when fitted with (a) an ARIMA(0,1,1) and (b) an
ARIMA(0,0,0) model with a regressor.
no autocorrelation in the process, and the most appropriate ARIMA model is white noise. Figure 3.3b shows the residual errors for this model fit.
If we want to be robust to the presence of changes in mean, whilst correctly modelling the autocorrelation structure of the data, it is important to consider changes in mean as a part of the preprocessing step of building a model used to forecast. As such, here we outline a changepoint preprocessing method which detects changes in mean and incorporates these into the time series model.
3.2.1
The Model
As an introduction to building our model for forecasting, we first test for any changes in mean. To do this, we take a penalized likelihood approach to changepoint detection, as described in Section 2.4. In this setting, we replace the cost function C(·), in equation (2.4), with twice the negative log-likelihood for a Gaussian distribution with common variance and segment specific mean.
The second component of equation (2.4) is the penalty used to prevent over-fitting to the mean of the data. We are assuming independent Gaussian observations. However,
0.00 0.01 0.02 0.03 0.04 0.05 0.0 0.2 0.4 0.6 0.8 1.0
False Positive Rate
T rue P ositiv e Rate µ=0.25 µ=0.5 µ=0.75 µ=1
Figure 3.4: Receiver operating curves for (solid line) i.i.d. normal data and (dashed line) correlated normal data (Autoregressive data with parameter 0.8) which exhibits a change in mean from zero to a new mean levelµ.
in a forecasting setting our data will most likely contain autocorrelation structure. Despite this, the algorithm is still effective at locating changes in mean (Lavielle and Moulines, 2000).
Figure 3.4 shows the receiver operating curves (ROCs) for detecting a change in mean both with and without autocorrelation. In this simple example we can see that when there is autocorrelation present, we have increased power to detect changes. However, this results in an increased false positive rate. This inflation of the type I error rate can also be seen in Lund et al. (2007). To remedy this, practically we inflate the standard penalty chosen as suggested by Lavielle and Moulines (2000).
3.2.2
Forecasting
Once we have detected changes in mean, we can incorporate them into our time series model using external regressors. Algorithm 1 provides pseudo code for forming a matrix of regressors based upon the changepoint locations. Having attained a matrix of regressors representing the level changes, we can fit a multiple linear regression
between these and the data in order to remove the effect of the level shifts. This can be done independently of the time series model, in which case we would fit the time series model to the residuals of the linear regression, or it can be done as part of the modelling process.
Algorithm 1: Incorporating mean changes into forecasts Data: Time series Y = (y1, . . . , yn), xi ∈R
Result: Matrix of external regressors representing mean changes to be used for both model fitting and forecasting.
1 Letτ0 = 0 andτm+1 =m and detect changes in mean τj for j = 1, . . . , m.; 2 if m = 0 then 3 V =N U LL; 4 Vout =N U LL; 5 else 6 V ∈Rm×n; 7 for j ∈[1, . . . , m] do 8 vj,i = 1 i∈(τj−1, τj], 0 otherwise. 9 end 10 Vout = 0m×1; 11 end 12 return V, Vout
In Section 3.4 we test this approach in a simulation study. In the next section, we turn our attention to an alternative approach to using changepoints to improve forecasts.