Changes in Dependence Structure

As discussed in Section 2.1, traditional univariate changepoint models typically as- sume that the observations of a time series occur independently over time. However, modern changepoint detection methodology has been providing more consideration to cases where there is dependence between observations. Indeed, changepoint methods have been developed which can not only incorporate dependence into the model, but are actively aiming to detect changes within the second-order structure of the series. Popular approaches adopted by such methods include: (i) a time-domain treat- ment involving the traditional likelihood of the series, (ii) utilising an approximation to the traditional likelihood called ‘Whittle’s likelihood’ which allows for a frequency-

domain analysis, or (iii) considering non-parametric statistics. We consider methods which employ each of these approaches in turn.

A likelihood-based approach is arguably the most common approach to detect- ing changes in second-order structure. Davis et al. (2006), Gombay (2008), Killick et al. (2010) and Fryzlewicz and Subba Rao (2014) all propose procedures based on calculating the traditional likelihood of dependent time series. The Auto-PARM method of Davis et al. (2006) uses the traditional likelihood-based minimum descrip- tion length (MDL) of an AR(p) process as a penalised cost function, and use a genetic algorithm to estimate the number and locations of changes in the autoregressive struc- ture. Similarly, Gombay (2008) considers the detection of changes in any combination of parameters of a p-order autoregressive process via a hypothesis testing procedure, where the test statistics are based on the likelihood of the process. Killick et al. (2013) also utilise the traditional likelihood, but instead model the observations as a Locally Stationary Wavelet (LSW) process (Nason et al., 2000), referring to this as the Wavelet Likelihood. They use this likelihood as a test statistic in a binary segmentation framework, and use a graphical data-driven method to determine the number of changepoints (rather than a specific penalty). Fryzlewicz and Subba Rao (2014) also use binary segmentation with a likelihood-based framework, but instead detect multiple changes in ARCH and GARCH processes.

Whittle’s likelihood approximates the traditional likelihood in terms of the spectral density of the series. Therefore, this quantity has allowed for the detection of changes in the second-order structure of univariate time series. We provide an in-depth exami- nation of Whittle’s likelihood and its application to changepoint detection in Chapter 3. Notable works within the changepoint literature which employ Whittle’s likelihood include those of Lavielle and Lude˜na (2000), Hsu and Kuan (2001), Yamaguchi (2011) and Yau and Davis (2012).

Lavielle and Lude˜na (2000) utilises Whittle’s likelihood in a penalised cost func- tion framework to detect changes in the spectral density of a time series. However, the penalty function assumed in their model is required to be linear in the number of changepoints. While this is theoretically interesting, this requirement does not

CHAPTER 2. CHANGEPOINT DETECTION AND TIME SERIES MODELS 42 allow for the usage of popular non-linear penalties such as the Minimum Description Length (see Chapter 3 for more details). Hsu and Kuan (2001), Yamaguchi (2011) and Yau and Davis (2012) all consider the context of changes where long-memory may be present. In the case of Hsu and Kuan and Yau and Davis, interest lies in distinguishing whether a given series follows a long-memory model or whether it is a short-memory process with an abrupt change in the dependence structure. The problem considered by Yamaguchi (2011) is the estimation of a changepoint in the long-memory parameter of an Autoregressive Fractionally Integrated Moving Average (ARFIMA) process. Such a process is a generalisation of an ARMA process which allows for fractional differencing, see Hosking (1981) for more details. In each case, Whittle’s likelihood approximation is used to evaluate the suitability of a given model. Giraitis et al. (1996), Ombao et al. (2001) and Cho and Fryzlewicz (2012) detect second-order changepoints using non-parametric approaches. Giraitis et al. (1996) use Kolmogorov-Smirnov-type statistics to test for changes in the distribution of de- pendent data. Ombao et al. (2001) propose a new set of bases which can be used to decompose a time series, with this decomposition then being used in a non-parametric test statistic to detect second-order changes. However, this suffers from its require- ment that changes must occur at dyadic time-points. In a similar manner to Killick et al. (2013), Cho and Fryzlewicz (2012) model observations using the Locally Sta- tionary Wavelet framework, but instead search for changes in the mean of the wavelet coefficients using a non-parametric test statistic in a binary segmentation procedure. These changes in mean in the wavelet coefficients correspond to changes in the second- order structure of the original series.

Killick et al. (2013) demonstrate that their approach (termed ‘WL’) out-performs the method of Cho and Fryzlewicz (2012) in terms of quality of solutions. This may be due to the assumption made by Cho and Fryzlewicz that the variance of the summary statistic is constant across different segments, which can be difficult to establish in practice. They also show that while the Auto-PARM method of Davis et al. (2006) estimates the correct number of changepoints more often than WL, the changepoint locations estimated by WL are more accurate than those estimated by Auto-PARM.

As has been seen, there exists a range of methods for detecting second-order changes. While there are a number of methods which utilise Whittle’s likelihood, the majority of these consider long-memory models, and the method available for short-memory models is impractical. Therefore, in Chapter 3 we propose methodol- ogy which employs Whittle’s likelihood to detect changes in short-memory time series models which can be easily implemented in practice. This practicality is demon- strated through application to a substantive dataset arising from acoustic sensing observations.

Chapter 3

Detecting Changes in

Second-Order Structure: An

Application to Acoustic Sensing

Data

3.1

Introduction to Acoustic Sensing Data

In the previous chapter, we highlighted the development of various approaches to detecting changes within piecewise second-order stationary time series. Simulation studies reported by Davis et al. (2006), Cho and Fryzlewicz (2012) and Killick et al. (2013) have shown that many of these approaches have broadly good performance across a wide range of different scenarios. However, it is well-known that several of these methods are also computationally intensive. Consider, for example, the wavelet-likelihood approach of Killick et al. (2013) which (as we shall see later) is On4(log n)2. Such significant computation can prove prohibitive for even moder-

ately long time series or applications where many time series need to be processed on a regular basis. Acoustic sensing signals, such as those becoming commonly ob- tained in the oil and gas industry, provide an example of such an application. Within

this chapter we therefore seek to explore which approach, of the various available in the literature, provides the best combination of changepoint detection accuracy and speed, and investigate the potential for their application to acoustic sensing data.

Acoustic sensing is the practice of measuring and quantifying the vibrations which are travelling through some medium, typically the ground. Within oil exploration and production, such vibrations are measured by lining the well with a fibre-optic cable. When vibrations occur in the medium they pass through the fibre-optic cable, inducing a change in the intensity of the reflection of the pulses of light being passed through the cable. These pulses of light are produced at a very high rate, often as high as 10kHz, allowing for the ‘real-time’ monitoring of these vibrations to identify features of interest in the well (e.g. the composition of the oil and gas, or areas where the gradient of the piping changes), or mapping of the geology of the local environment. The characteristics of such vibrations means that the observations are generally dependent in time. For further discussion of acoustic sensing in the oil and gas industry see, for example, Van der Horst et al. (2014) and Silkina (2014).

In addition to physical features being visible within these vibration measurements, there occasionally exists error features within the series. Such errors may be due to an external disturbance of the fibre-optic cable or some other (unknown) factor. We are advised by engineers that such error features manifest as sudden changes in the second-order structure of the time series. Typically, error features induced by these disturbances occur at all observed locations of the well. The magnitude of the disturbances relative to the true features is such that it is only necessary for a single channel to be analysed in order to detect the disturbance. Figure 3.1.1 presents three examples of acoustic sensing time series from one particular type of well. Figure 3.1.1(a) shows data without any error effects, as demonstrated by the visibly stationary nature of the series. Conversely, Figures 3.1.1(b) and 3.1.1(c) both demonstrate instances of disturbance, which are clearly illustrated by the abrupt increases in vibration, followed by a period of increased activity, before returning to a low level of vibrations.

CHAPTER 3. DETECTING CHANGES IN SECOND-ORDER STRUCTURE 46 Time V alue 0 200 400 600 800 1000 −1.5 −1.0 −0.5 0.0 0.5 1.0

3.1.1(a): Acoustic sensing time series without error features. Time V alue 0 200 400 600 800 1000 −1.5 −1.0 −0.5 0.0 0.5 1.0

3.1.1(b): Acoustic sensing time series contain- ing error features caused by an exter-

nal disturbance. Time V alue 0 200 400 600 800 1000 −1.5 −1.0 −0.5 0.0 0.5 1.0

3.1.1(c): Acoustic sensing time series contain- ing error features caused by an exter-

nal disturbance.

Figure 3.1.1: Three examples of acoustic sensing time series obtained from one particular type of well. The error features are present in the second

and third series.

is capable of identifying second-order changes, such as the error features described, through the utilisation of Whittle’s likelihood approximation. This is a popular tool for analysing time series in the stationary context. We demonstrate that our method is pragmatically appropriate and draw comparisons with other leading second-order changepoint methods. The presented methodology is applied to substantive acoustic sensing data where it is shown that the locations of detected changepoints correspond with occurrences of error features.

Whittle’s likelihood approximation and examines how it can be used in a penalised likelihood framework for detecting changes in second-order structure. Section 3.3 com- pares the performance of a second-order changepoint detection method using Whit- tle’s (approximate) likelihood against different approaches which use exact likelihood based formulations. A selection of acoustic sensing data is analysed using the pro- posed Whittle likelihood based method in Section 3.4, and concluding remarks are presented in Section 3.5.