http://dx.doi.org/10.4236/am.2013.412221
A Permutation Test for Unit Root in an Autoregressive
Model
Jiexiang Li1, Lanh Tran2, Sa-aat Niwitpong3
1Department of Mathematics, College of Charleston, Charleston, USA 2Department of Statistics, Indiana University, Bloomington, USA
3Department of Applied Statistics, King Mongkut’s University of Technology, Bangkok, Thailand
Email: [email protected], [email protected], [email protected]
Received September 23, 2013; revised October 23, 2013; accepted October 30, 2013
Copyright © 2013 Jiexiang Li et al. This is an open access article distributed under the Creative Commons Attribution License,
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
ABSTRACT
A permutation test (based on a finite random sample of permutations) for unit root in an autoregressive process is con- sidered. The test can easily be carried out in practice and the proposed permutation test is neither limited to large sample sizes nor normal white noises. Simulations show that the power of the permutation test is reasonable when sample sizes are small or when the white noises have a heavy tailed distribution. The test is shown to be consistent.
Keywords: Permutation Test; Autoregressive; Nonstationary
1. Introduction
Let 1 2 1 be observations of the real
valued autoregressive model , , , n
Y Y Y n1
1 ,
t t
Y aY et
for and t is a sequence of independent
identically distributed random variables with mean zero and variance
0 <a1 e
2
and Y10. Tests of
0: 1 versus A: 0 < < 1,
H a H a
are often referred to as tests for unit root. The hypothesis that is of interest in applications because it corresponds to the hypothesis that it is appropriate to transform the times series by difference. In [1] and [2], the authors derived the limit distribution of the statistic
with 1
a
ˆ1
n a
1
1 1
2
1 1
2 2
ˆ n t n t
i t
a Y Y Y
tunder the unit root assumption . However, [1] and [2] are limited to large sample sizes or normal white noises. When sample sizes are small and white noises are from distributions with heavy tails, to test for the pres- ence of unit root, we can use the permutation test pro- posed in the paper.
1
a
Under H0, t is not stationary and the variance of
is t
Y
t
Y 2. When H0 is true, is sometimes called a Yt
random walk and XtYt1Yt , are
independent identically distributed r.v.’s. In Economics, it is important to characterize the velocity, or stock price as a random walk. Another way of phrasing 0
1, 2, ,
t n
H is that
whatever determinants of velocity and their individual stochastic structure may be, their combined effect is such that successive changes in velocity are essentially inde- pendent. This would imply that of the past history avail- able at any given date only the current observation is relevant for prediction. Testing for unit root is equivalent to testing for serial independence in sequence X1, , Xn.
For some literature on tests for serial dependence, see [3] and [4] and the references therein. We define
1 1 1
, n
n i i
i
T X X
and for each random permutation of the vector
X1, , Xn
, say,
X1, , Xn
, denote
1 1 1
. n
n i i
i
T X X
If H0 holds then
X1, , Xn
e2, , en1
. Thedistribution of a random vector of i.i.d. random variables is invariant under any permutation of its coordinates. Thus Tn and Tn have the same distribution if H0 is
statistic Tn under permutations when H0 holds, we
propose a permutation test for unit root using n as our
pivot test statistic. This test is easy to perform with a computer and the test makes little assumptions on the probability distribution of the white noise and also it works for small samples. Construction of this test will be presented in Section 2. The consistency of our test is shown in Section 3. A simulation study of our test is provided in Section 4. Time series of velocity of money observed between 1869 and 1960 is investigated in Sec- tion 5.
T
2. Steps Used in Our Permutation Test
We assume that the white noise et is a sequence of
independent identically distributed random variables with mean zero and variance 2. In addition,
4 <t
E e .
We will often write Tn simply as for brevity.
Note that T is more likely to be negative under A T
H
(see Lemma 3.2 below). To summarize, the permutation test is carried out as follows.
1) Set a predetermined level . Permute the observations 1 2
n
X X, , , Xn
. There are a total of permutations. For each permutation, compute the Tstatistic. Under 0
!
n
H , the T statistics have the same
probability distribution for all of the permutations. The statistic computed from the observations (not permuted) is referred to as .
!
n T
obs
2) Compute the -value as the proportion of T's
less than or equal to , that is,
T p
obs
T
number of
-value Tobs.
!
T s p
n
Conclude that the test is statistically significant if the computed p-value is less than or equal to .
This test is limited by prohibitive calculation and hard to carry out if is a large number. Instead of using all permutations to compute the , we obtain a random sample of permutations and then carry out the test as follows:
n
!
n p-value
R
1) Set a predetermined level . Compute the T
statistic for each of the R sampled permutation. 2) Compute the -value as the proportion of T's
less than or equal to , that is,
p
obs
T
number of
-value T s Tobs.
p
R
Conclude that the test is statistically significant if the computed p-value is less than or equal to .
The approximate -value is now equal to the fraction of 's that are less than or equal to obs. The theory of
the binomial distribution tells us that the approximate value has about a 95% chance of being within
p
T T
2 p 1 p
R
of the true -value. We will denote the permutation test based on a random sample of permutations by
p
R R.
Remark. Consider the lag one autocorrelation
1
1 1
1 2
1
. n
i i
i n
i i
X X X X
r
X X
Using is “almost” equivalent to using 1 to
perform our test. We leave it to the reader to verify this.
T r
3. Consistency of
RConsistency of hypothesis tests is a desirable property. In this section, we will show that the permutation test based on random sampling permutations is consistent, that is, the probability of correctly rejecting the null hypothe- sis
R
0
H tends to 1 as the sample size goes to infinity
when 0 < < 1a . In other words,
Theorem 3.1 Suppose Ha is an arbitrary simple
hypothesis that the autoregressive parameter is between 0 and 1, that is
a
a A
H H . Then
Reject 0
1a
H
P H
as n .
Lemma 3.1 Under HA, for any integer m1,
2
1 1
1 1 1 2 2 .
m m m
m
E X X a a a
a
Proof. We can write Yt as
0
. u t
u
Y a e
t u (1) Using (1), it is easily seen that under HA, the AR(1)process Yt has mean zero with
22Var ,
1 t
Y
a
and
1 22 .m t t m
E Y Y a
a
Clearly,
1 1 m
2 2 m 2 1m 1 2 m 1 1
.E X X E Y Y Y Y Y Y Y Ym In particular,
1 2
2
1 , 1
a E X X
a
2 2 1 21 1 2 2
0 0 0 0
2 4 4 1 2 1 2 4 2 2 4
1
0 0 0 1
2 2 2
4 4
4 2 2 4
2 4 2 4 2 2 2
1 1 1
1
. 1
1
u v p q
u v p q
u v p q
u u v
u
u u v u u v u
t
E Y Y
E a e a e a e a e
a E e a a a a
a a a
E e
a a a
a a a u v
(2) Lemma 3.2 Under HA, T nn converges in proba-bility to
2 1 1 1 2 a E X X
a
Proof. Under HA, time series is stationary and so
is time series t
Y
1 t t
X X . Thus
1 2
,1
n
T
E E X n
X
it suffices to show that
Var 0. 1 n T n
1 2 1 1
1 1
1
1 2 1 2 1
2
1 2 1 2 2 3
1 2 3 4
1 2
1 2 1 1 2
4
Var
1 Var 2 Cov ,
1 Var 2 Cov ,
1 Var 2 2 Cov ,
2 3 Cov ,
2 .
n
i i j j
i j n n j j j n j j j T
n X X X X X X
n X X n j X X X X
n X X n X X X X n X X X X
n j E X X X X E X X
(3) By (3), it is sufficient to show that
2 1
1 2 1 1 2
4
2 0
1 n
j j
j n j E X X X X E X X
n
as n .
For , consider . Following the proof of (2), we have
4
j E X X X X
1 2 j j1
1 2 1
2 3 2 2 2 1 2
4 4
2 2 2 1 2 3
4 4
2 3 4
4 2
2
2 3 2 2 2 1 2
4 4
2 1 2 2 2 3
4 4
2 4 2 3 2 2 2 1 2
4 2
2
3 3 2
1
3 3
1
1 4 6 4
1
3 9 9 6
1
9 9 3
1
2 8 12 8 2 .
1
j j
j j j j
t
j j j
t
j j j j
j j j
j j j j j
E X X X X
a a a a
E e a
a a a
E e a
a a a a a
a a a a
a
a a a
a
a a a a a
a
It is not hard to see that
1
2 3 2 2 2 1 2
2 4
1
2 1 2 2 2 3
2 4
1
3 3 2
1
1 3 3
1 n
0
j j j
j
n
j j j
j
n j a a a a
n
n j a a a
n
j and
2 3 4
1
2
2 2
4
2 4 2 3 2 2 2 1 2
1 2 2 2 4 2 1 2 2 4
1 4 6 4
1 1
2 8 12 8 2
1 1 1 0 1 1 n j
j j j j
n
j
n
j
a a a a n j
n a
a a a a a
n j n a a n j n a
jas n . This completes the proof.
On the left of Figures 1 and 2, it shows the long run behavior of T nn based on 10,000 simulations from
AR(1) model with white noise from normal (0,1) and uniform (−1,1) respectively.
Lemma 3.3 Under HA, as
.
1 2
0E X X
n
Proof. For a random permutation, R , of
1, 2, n.
X X X
1 2
1 1
i j
P X X X X
n n
for any 1 i j n.
1 2 1 2 1
1 , , 2 . 1 n i j i j n
E X X E E X X X X
For i< j, by stationarity and Lemma 3.1,
1 1
2
1 1
2 2 .
1
i j j i
j i j i j i
E X X E X X
a a a a
(5)
From (4) and (5),
1 2
22
1 2
. 1 1
n
na n a E X X
n n a
(6)
The proof follows from (6).
Lemma 3.4 Under HA, as
.
1 2 3 4
0E X X X X
n
Proof.
1 2 3 4
1
4!
.
1 2 3
i j k l
i j k l n
E X X X X E X X X X
n n n n
(7)
Note
1 1 1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1
1 1 1 1 1
1 1 1
1 1
i j k l
i j k l i j k l
i j k l i j k l
i j k l i j k l
i j k l i j k l
i j k l i j k l
i j k l i j k l
i j k l
E X X X X
E Y Y Y Y E Y Y Y Y E Y Y Y Y E Y Y Y Y E Y Y Y Y E Y Y Y Y E Y Y Y Y E Y Y Y Y E Y Y Y Y E Y Y Y Y E Y Y Y Y E Y Y Y Y E Y Y Y Y
1
1 .
i j k l
i j k l i j k l
E Y Y Y Y E Y Y Y Y E Y Y Y Y
After lengthy calculation of , we know that to show (7) converges to zero when goes to infinity, it is sufficient to show
i j k l
E X X X X n
1 2
4 3 1 < < <
3 2 1
4 3 1 < < <
2 3 3
3 3
0;
j k l j l k j l k j l k
i i j k l n
j l k j l k j l k
i i j k l n
a a a a
n a
a a a
n a
3
1 1
4 1 < < <
2 2
4 1 < < <
6 4 4
0;
j l k i j l k i j l k i
i j k l n
j l k i j l k i
i j k l n
a a a
n
a a
n
1 1
4 1 < < <
2 2
4 1 < < <
12 8 8
2 2 0,
k l j i k l j i k l j i
i j k l n
k l j i k l j i
i j k l n
a a a
n
a a
n
which are easy to see.
Lemma 3.5 Under HA, Tn n converges to 0 in
probability for all 1 R.
The proof follows from (3), Lemma 3.3 and 3.4. On the right of Figures 1 and 2, it shows the long run behavior of Tn n based on 10,000 simulations from
AR(1) model with white noise from normal (0,1) and uniform (−1,1) respectively.
Proof of Theorem 3.1. Clearly, under HA,
, for all 1
, for all 1 , n n
n n
P T T R
P T n T n R
[image:4.595.315.536.492.705.2]
Figure 1. Illustrations of Lemma 3.2 and Lemma 3.5 with normal (0,1) noise and a .8.
Figure 2. Illustrations of Lemma 3.2 and Lemma 3.5 with
[image:4.595.77.264.540.731.2]which tends to 1 by Lemma 3.2 and Lemma 3.5. In particular, the probability of rejecting H0 under HA
tends to 1.
4. Simulation Study
Consider the model t t1 t , , 1 where the t has contaminated normal distri-
[image:5.595.312.534.87.306.2] [image:5.595.312.534.342.561.2]bution. Note the contaminated normal observations were generated in the following way: 70% of the time an observation is generated from a standard normal distri- bution while 30% of the time it is generated from a normal distribution with mean 0 and standard deviation 25. One thousand samples of size , 25, 50, 100, 250, 500 were generated for , 0.4, 0.6, 0.8, 1. Permutation tests based on all permutations when sam- ples are small or randomly selected 1000 permutations for unit root were applied to each sample at the sig- nificance level 0.05. The power of the test is tabulated in Table 1 based on 1000 simulated tests for , 25, 50, 100, 250, 500 and , 0.4, 0.6, 0.8. From the table, it is easy to see that the power gets closer to 1 when the sample size increases and this demonstrates the con- sistency of the proposed permutation test.
Y aY e
n
0.2
a
0.2
2, , 1
t n
5
5
n
0
Y e
a
5. An Example
Reference [5] studied the stochastic structure of velocity in order to determine whether there is a statistical basis for extrapolative prediction. Noting that the velocity of money is defined as the ratio of national income to the stock of money. In the paper they conclude that the logs of the velocity series constructed in [6] are well char- acterized as a simple random walk. As preliminary analysis, we look at the time series plot of t, the cen-
tered logs of velocity. The pattern in the time series plot is typical of a nonstationary series of the sort which displays no affinity for a mean value. We also note that the autocorrelations of centered logs of velocity series are very large and decline slowly with increasing lag (Figure 3). Now let us look at the time series plot of t
Y
X ,
the first differences of centered logs of velocity, and autocorrelations of Xt (Figure 4). Judging from the
[image:5.595.58.287.629.733.2]time series plot and autocorrelations of time series Xt,
Table 1. Power of our permutation test.
n 5 25 50 100 250 500
a = 0.2 0.081 0.639 0.905 0.997 1 1
a = 0.4 0.051 0.398 0.71 00.933 1 1
a = 0.6 0. 047 0.19 0.342 0.652 0.954 0.998
a = 0.8 0.045 0.09 0.147 0.196 0.449 0.706
a = 1 0.054 0.046 0.047 0.044 0.048 0.055
Figure 3. Time series Yt and its autocorrelations.
Figure 4. Time series Xt and its autocorrelations. there seems no significant dependence in the time series
t
X . Moving on to the formal analysis, we can see that it
is reasonable to fit model t t1 t to the centered
logs. With an application of permutation test on the centered logs, we obtain the test statistic
Y aY e
0.04
obs
T ,
and the p-value = 0.7674. The null hypothesis is not
rejected at any reasonable level.
REFERENCES
[2] D. A. Dickey, “Estimation and Hypothesis Testing in Nonstationary Time Series,” Ph.D. Dissertation, Iowa State University, Ames, 1976.
[3] N. H. Chan and L. T. Tran, “Nonparametric Tests for Serial Dependence,” Journal of Time Series Analysis, Vol.
13, No. 1, 1992, pp. 19-28.
http://dx.doi.org/10.1111/j.1467-9892.1992.tb00092.x
[4] H. J. Skaug and D. Dad Tjóstheim, “A Nonparametric Test of Serial Independence Based on the Empirical Dis-
tribution Function. Consistent Nonparametric Multiple Regression: The Fixed Design Case,” Biometrika, Vol. 3, 1988, pp. 591-602.
[5] J. P. Gould and C. R. Nelson, “The Stochastic Structure of the Velocity of Money,” The American Economic Re- view, Vol. 64, No. 3, 1974, pp. 405-417.
