Autoregressive Processes

One area in which changepoint methods have been applied and warrants discussion is detecting changes in autoregressive (AR) processes. Autoregressive processes are random processes which incorporate dependence between values at neighbouring time points and have been used to model speech signal (Davis et al., 2006), EEG (Chan et al., 2013) and earthquake (Adak, 1998) data as well as in other areas.

An autoregressive process of order p, denoted as AR(p), is defined as

Yt= ψ0+ p

X

i=1

ψiYt−i+ σ2Zt,

where ψ0, . . . , ψp, σ are the parameters of the model and Zt is white noise. We then aim to detect an

CHAPTER 2. LITERATURE REVIEW 25

process is a change in either ψ0 or σ (or both) and corresponds to a change in mean or variance (or

both).

Detecting changes in autoregressive processes and the related moving average models, ARMA and ARIMA models have been looked at by a number of authors. For detecting a single change in autore- gressive models Basseville and Benveniste (1983) compare the two models (with or without a change) by the use of a test statistic. They suggest using either the log-likelihood ratio test or the Kullback-Leibler divergence (a non symmetric measure of the difference between two probability distributions).

For multiple changepoint detection in AR parameters Hamilton (1990) use an expectation-maximisation (E-M) algorithm. They model the changes in the parameters as outcomes of a discrete valued Markov process. A more general approach for detecting changes in stationary time series is developed and applied to autoregressive processes in Adak (1998). The approach taken here is to estimate the time dependent spectrum use a binary tree to perform the segmentation, to speed the method up they suggest only searching for segments of dyadic length. Another general approach is outlined in Shao and Zhang (2010) where a self normalisation based Kolmogorov-Smirnov test is proposed. This test statistic is asymptoti- cally distribution free and is applied to detecting changes in second order structures and more specifically in an AR(1) process.

The LASSO (Section 2.2.4) is extended for AR processes in Chan et al. (2013). They show that whilst the LASSO overestimates the number of true changepoints, a subset of the estimated set can be selected by a second step selection procedure. They then show that this subset correctly estimates the number and location of the changes with probability tending to one.

The PELT algorithm (discussed in Section 2.2.2) is applied to autoregressive processes in Killick et al. (2012). They look for changes in the autocovariance of AR processes and compare against the approach of Davis et al. (2006) (discussed below). Changes in the autocovariance of AR processes is also considered in Korkas and Fryzlewicz (2015), where they use the WBS algorithm and detect changes using wavelet based non-stationary time series techniques.

ARCH (Autoregressive Conditionally Heteroscedastic) processes are processes where the conditional variance evolves according to an autoregressive process. Changes in ARCH processes are considered by Fryzlewicz and Subba Rao (2014) where they build on the Binary Segmentation algorithm to find changepoints in the FTSE 100 index. Their method, BASTA (Binary Segmentation for Transformed

CHAPTER 2. LITERATURE REVIEW 26

Autoregressive Conditional Heteroscedasticity), first transforms the data to decorrelate it and lighten the tails, and then performs Binary Segmentation to find the changes.

Further work on detecting changepoints in ARCH models has been widely looked at in the econo- metrics literature where they are often referred to as “switching ARCH models”. This is first looked at in Cai (1994) where a switching ARCH model is fitted by combining an ARCH process with a switching regime Markov process. Further work on this can be found in, for example, Haas et al. (2004) and Pelletier (2006) and the references therein.

Detecting Changes in AR Models using the Minimum Description Length

One algorithm of note for detecting changes in autoregressive processes is presented in Davis et al. (2006) and minimises a cost based on the minimum description length.

The minimum description length (MDL) principle was first introduced in Rissanen (1989). The idea of MDL is to choose the model which stores the data in the least possible space, or with the smallest code length. Techniques for calculating the MDL for various models by approximating the code length are given by Rissanen (1989).

Using the techniques from Rissanen (1989), Davis et al. (2006) calculate the MDL for a changepoint model where each segment is modelled by an autoregressive process of unknown order. This cost is as follows M DL(m, τ1, . . . , τm, p1, . . . , pm+1) = log m + (m + 1) log n + m+1 X j=1  log pj+ pj+ 2 2 log sj+ sj 2 log(2π ˆσ 2 j)  ,

where p1, . . . , pm+1 are the orders of the AR processes for each segment, sj = τj− τj−1 the length of

segment j (for j = 1, . . . , m + 1) and ˆσ2

j is the Yule-Walker estimate of σj2.

The best fitting model is then chosen as the model which minimises the MDL. As the search space is large, Davis et al. (2006) suggest performing the minimisation via a genetic algorithm. Genetic algorithms are heuristics and hence do not solve the minimisation problem exactly. Despite this, Davis et al. (2006) show that the method performs well in their simulations however it should be noted that they only compare to other heuristic methods.

CHAPTER 2. LITERATURE REVIEW 27