CHAPTER 4: NONPARAMETRIC MULTIVARIATE CHANGE POINT ⋅ ⋅ ⋅
4.3 Nonparametric Multivariate Change Point
4.3.1 Problem Set Up
As noted earlier, PA, and therefore EE, can aect BG after a certain amount of time, potentially an hour or more. The relationship can be very dynamic in nature and therefore we believe that, BG will depend on lagged values of EE. Moreover, we want to know how EE and its lagged values aect BG an hour into the future and its consecutive values after 5 and 10 minutes. Hence, we want to look at the relationship between a vector of lagged values of EE and a vector of future values of BG. Thus, our problem is multivariate in nature and we wish to estimate a change point in the relationship between these 2 vectors. We develop a method to solve this problem. The set up is described below.
We are interested in a sequence of a pair of multivariate vectors (Yt, Xt) for t = 1, . . . , T, where X ∈ Rp and Y ∈ Rq. The pair (Yt, Xt) are observed sequentially along a dimension denoted by the index t. In our context t can denote time, and Yt
can be several values of BG and Xt can be several values of EE, at timet. The change
point problem that we are interested is characterized in the following data generating mechanism:
Yt=f1(Xt, ηt) for t=1, . . . , τ,
Yt=f2(Xt, ηt) for t=τ+1, . . . , T,
(4.1)
where ηt and Xt are i.d.d for each t, and are independent of each other. Thus, Yt
depends on Xt through a function f1. However, as t advances, a point τ is reached
this change-point actually exists. The hypotheses that we are interested in are
H0∶f1=f2, HA∶f1≠f2,
(4.2)
and note that under the null hypothesis τ vanishes. Possible approaches in testing (4.2) would require estimating f2 and f2. It can be dicult to explicitly model the multivariate relationship between randomX andY parametrically; this becomes more dicult if Y is categorical, or if each element of Y is categorical of dierent type, i.e. ordinal, nominal or count; for estimating the change-point, a model is estimated at each (time) point and this can be computationally intensive; and there are approaches that model the change-point nonparametrically for the joint distribution of(X, Y), but not for the relationship between Y and X, and thus this approach is too general for our purpose.
In the current research, we avoid the complications of modelingf1andf2. Instead, we will test (4.2) by creating a test statistic based on the distance covariance (DC). The DC statistic, denoted by V2(X, Y) for the DC between X and Y, can evaluate how strong the statistical dependence between two random vectors is. The larger the value the stronger the dependence. A value of DC of exactly 0 is equivalent to statistical independence. In the current context, we will construct two DCs between two random vectors, one before and one after a given potential change point τ∗. If the dierence between these two DCs is large for this specicτ∗, then this will provide evidence that τ∗ constitutes a change-point. We will look at all such dierences, times a rate, across all possibleτ's and select the one position that maximizes this dierence and denoted this τˆ; this will be our estimator of the change-point.
change point τ∗, if the distribution of X changes while its relationship to Y remains intact, then this will generate slight dierences in the sample DC that are not at- tributable to changes in the relationship betweenY and X before and after τ∗. Hence, for the sake of stability, while evaluating if a given τ∗ constitutes a true change point, the marginals of all the sample data of X and Y before τ∗ will be transformed to be discrete uniform. The same will be done for all data marginals of X and Y after τ∗. Then, for this τ∗, the dierence in DC will be assessed by using this uniformly trans- formed data instead of the original data. The uniform transformation will be performed anew for each potential change point andτˆwill be chosen to be the one that maximizes
the dierence in DC. This will allow the estimator τˆ to be less sensitive to changes in
marginal distribution that are not related to changes in the relationship of X and Y. We will give more details of the transformation in Section 4.3.4.
Once an estimate τˆof the change point is attained, it is important to evaluate if
it constitutes a true change point. The reason is that change point estimators always provide a change point regardless of whether one exists or not. Then, a test of the existence of the change point should be performed. One can test a similar set of hypotheses as (4.2) to accomplish this by testing instead
H0∶V2(X1, Y1) = ⋯ =V2(XT, YT),
HA∶⋯ =V2(Xτ, Yτ) ≠V2(Xτ+1, Yτ+1) = ⋯,
(4.3)
for some unknown τ. Our main assumption is that, if the change-points described in (4.1) exists, then V2(X
τ, Yτ) ≠ V2(Xτ+1, Yτ+1), meaning that, if a change-point exist then, that change-point will be accompanied with a dierence in DC before and after the change-point.
t=1, . . . , T and recalculating the test statistic many times to create a null distribution. Among the virtues of this estimation and testing procedure are that no form forf1 and f2 need ever be specied, that the test statistics is easy to compute and requires no regularization, and that it can accommodate any kind of data of any dimension.
4.3.2 Distance Covariance
Distance covariance was developed by Székely et al. (2007), Székely et al. (2009), and Székely and Rizzo (2013). For random variables X ∈ Rp and Y ∈ Rq, let φx, φy and φx,y be the characteristic function of X, Y and (X, Y), respectively. Assume that E∣X∣p < ∞ and E∣Y∣q < ∞. Distance covariance (V) can be used to measure the dependence between X and Y through the distance
V2 (X, Y) =∣∣φx,y(t, s) −φx(t)φy(s)∣∣2 = ˆ Rp+q ∣φx,y(t, s) −φx(t)φy(s)∣2(cpcq∣t∣1p+p∣s∣1q+q)−1dtds with cd= π (1+d)/2
Γ((1+d)/2) and ∣ ⋅ ∣p is the Euclidean norm in Rp. IfX ⊥/ Y then V2(X, Y) will be greater than 0. otherwise if X⊥Y then it will be exactly 0. Distance variance can be dened as V2(X) = V2(X, X). The distance correlation between X and Y is the nonnegative numberDC(X, Y) dened by
DC(X, Y) = V
2(X, Y) √
V2(X)V2(Y)
if V 2(X)V2(Y) > 0 and equals 0 otherwise. The distance covariance statistic are dened as follows. For an observed random sample {(Xk, Yk) ∶ k =1, ..., n} from the
joint distribution of random vectors(X, Y) ∈Rp×Rq, dene akl=∣Xk−Xl∣p, a¯k⋅= 1 n n ∑ l=1 akl, ¯a⋅l= 1 n n ∑ k=1 akl, ¯ a⋅⋅= 1 n2 n ∑ k,l=1 akl, Akl=akl−¯ak⋅−¯a⋅l+¯a⋅⋅
for k, l =1, ..., n. Similarly, dene bkl = ∣Yk−Yl∣q and Bkl =bkl−¯bk⋅−¯b⋅l+¯b⋅⋅ for k, l =
1, ..., n. The empirical distance covariance Vn(X, Y) and distance variance Vn(X) are the nonnegative numbers dened by
V2 n(X, Y) = 1 n2 n ∑ k,l=1 AklBkl and Vn2(X) =Vn2(X, X) = 1 n2 n ∑ k,l=1 A2kl.
The empirical distance correlation DCn(X, Y)is dened as
DCn2(X, Y) = V 2 n(X, Y) √ V2 n(X)Vn2(Y) if V2
n(X)Vn2(Y) >0 and 0 otherwise. Both Vn2(X, Y) and DCn2(X, Y) are a.s. consis-
tent forV2(X, Y)and DC2(X, Y), respectively.