Autoregressive conditionally heteroscedastic models

Autoregressive Conditionally ~eteroscedasticModels

Sastry G. Pantula Department of Statistics North Carolina State University

Raleigh, NC 27695-8203

Key Words and Phrases:

Autoregression; heteroscedasticity; maximum

likelihood and generalized least squares estimators; asymptotic

normality.

Abstract

Two linear regression models, where the independent variables are

Sastry G. Pantula

North Carolina State University

1. Introduction:

Consider a process {Y

t} satisfying the equation Y

t • X~t~a + et (1.1)

where X : 1 x p is a known row vector and ~: p X 1 is a vector of unknown

~t

-parameters. Traditional econometric models assume that e

t is a sequence of uncorrelated (0,02) random variables and the conditional mean of Y

t

give~

the past is !t~' where !t may contain lagged values of Y

t • Under a symmetric loss function, the best forecast of Y

t, based on the past in~ormation, is the con-ditional mean of Y

t given the past and is denoted by E[YtIFt_lJ, where F

t-l • information up to time (t-l). Therefore for the model (1.1), the one step forecast error is

2 The unconditional variance of the one period forecast is given by 0 .

Note that

and

For the conventional econometric models, however, the conditional variance does not depend on F 1. For some processes one might expect better forecast

t-intervals if additional information from the past were allowed to affect the forecast variance.

realization is the bilinear model described by Granger and Anderson (1978).

Jones (1965), Granger and Anderson (1978), and Priestly (1978) considered

nonlinear time series models.

Engle (1982) proposed a class of models, called autoregressive conditionally

heteroscedastic (ARCH) models, for which both the conditional mean and the

. conditional variance of a time series are functions of the past behavior of the

time series. If the conditional density is normal, then a general expression

for the ARCH model is

where g and h are measurable. A special case we consider assumes that the mean

can be expressed as a linear combination of variables in the information set,

while the variance is a qth order weighted average of the squares of past

di.sturbances. More precisely,

where

and

e

t = Yt - X a-t ..

Assume that

aI' a

2, •.. ,

a

q are nonnegative and

a

O is positive. Although Yt

is conditionally normal, Engle (1982) established that the Y

t are not jointly

normal and the marginal distribution of Y

t is not normal. In this manuscript w"

Engle (1982) derived the moments of the let} process for q

=

1. In

section 2 we derive a representation for the let} process and use it to derive

the moments of the let} process. We also derive the ergodic properties of the

let} process. Engle (1982) also considered the maximum likelihood estimation of the parameters. He established that the information matrix is block diagonal,

. indicating that the maximum likelihood estimators of ~ and~are independent. He also indicated that the maximum likelihood estimators are asymptotically normal.

In section 3 we formally derive the asymptotic distribution of the maximum

likelihood estimators. We also derive the asymptotic properties of the least

squares and estimated generalized least squares estimators of ~ and ~.

2. Properties of the ARCH Models

The simplest ARCH model is the first order linear model given by

where

y

=

X a

t -t- (2.1)

and

(2.2)

F_t

-l = a-field generated by Ys and es' sSt-I, t • 1, 2, .•. It is assumed that

So

is strictly positive and

8

₁

is nonnegative. In the

following theorem we obtain a representation for the

{e~}

sequence which is

useful in deriving the properties of the model (2.1).

Theorem 2.1: Let let} be a sequence of random variables satisfying (2.2). Assume that

So

>

0 and 0 ~ 8₁

<

1. It is assumed that the process began

indefinitely in the past with a finite initial variance. Then,

2

~

t

t

Z2 .)

e

=

So

_{E B1(}

w a.s. ,

t

t=O

i=O t-~ (2.3)

where {Zt; t = 0, ±l, ±2, ... } is a sequence of normal independent (0,1)

Proof: Since e

t is conditionally normal, we get

where Zt is a standard normal random variable independent of e

t-l . Therefore,

2 _{Z2( 13} 2

e

t = t 0 + Slet_l )

2 2 2

= Zt [SO + SlZt_l(SO + _{6l e t _2)]}

j-l

_l

_l

₂

6j j-l 2 2 = 6 I: 6

1 ( 'IT Zt-i) + ( 'IT Zt-i)e t - j

0

_l=O

_i=O 1 _i=O

Note that,

00 l l 2 " )-1

<

E[ I: 6

1 (i'ITO Z .)] = (1 - 1>1 0 0 .

l=o

= t-l.

Therefore, by result (xi) of Chung (1974, p. 42),

co l l 2

I: 13 ('IT Z ) < co a.s . .

l=O

1 i=O t-i

Also, assuming that the process was initiated in the infinite past with a finite initial variance,

co

E [ I:

j=O Using the same argument,

and

co I:

j=O

"j ( j ;1 2 2

1>1 Z . )e .

<

co

i=O t-1. t-J a. s. ,

j j-1 Z2 ) 2

6 ( 11' e - 0 a • s ., as j + co • 1 i=O t-i t-j

Therefore,

2 co l l Z2 .)

e t = 60 I: _{61 ('IT} t-l. a.s.

0

l=O

i=O

2 A similar expression for e

t can be obtained in case q

>

1. Note that e

t has a sYmmetric distribution and that the sequence {et } is strictly stationary. The probability density function f of e

2

x

~ 2(SO+B_lu2 )

f(x) •

f

e

-00

We have not been able to obtain a solution for (2.4). We use (2.3) to obtain the moments of the {e

t} process. See also Engle (1982). Theorem 2.2: Let {e

t} be a sequence of random variables satisfying the conditions of Theorem 2.1. Then,

E[e2

t r

]

<

00

m_2r

=

i f and only i f

r

e •

B

r ~ (2j-l)

<

1 • r 1 j=l

Also, if e

<

1, then

r

Proof: Note that,

r-j

. and by Monotone convergence theorem (see Chung, 1974, p. 42),

Using Minkouski's inequality,

1 r

E[e~r]

_:a

₁₃r

f

n _Bj _{E j Z2r } r

]

0 .tim_n--

_..

_J=O.~ 1

i~O

t-i

• a

r

[n

_{-1 e;j+l)/rr}

.tim .~ 13

1 0 _{n-- J=O}

<

00 if

e

<

1 .

r

E[e;r] Sr co

sjr j zZr )

~

r

E( 'II'

0 _j=O 1 _i=O t-i Sr co -j 6( j+l)

= r _Sl

0 _j=O r

=

co if 6 ~ 1 r

Therefore, M

Zr is finite iff 6r

<

l. Now,

M

Zr = E[e;r]

.. E[ZZr<S _{+ Sl e2} )r]

t 0 t-1

r

(~) r-j sj -r

=

r

_So m_Zj6_rS_l

j=O J 1

)-1 r-l _(~) So r-j

,. 6 (1-6_{r r}

r

m_2j

₀

j=O J _Sl

Note that if 3S2

<

1, then 1

v(e~)

.. 204(1-3Si)-1 and

where

2

is a sequence uncorrelated (0,0 ) random variables.

We now give two lemmas that are useful in obtaining the ergodic properties of the {e

t} process.

Lemma 2.1: Let {U } be a sequence of F measurable random variables and

n n

F

n ~ Fn+l . Suppose there exists an integrable random variable U and a constant c such that

p[lu I

>xl

;:icp[lul

>xl.

n

-1 n -E,.

n _{k=l[Uk - E(UkIFk_l)1} .0.

If E[IUllog+lul1

<

~

or if {Uk,k

~

I} and {E(UkIFk_l),k

~

I} are strictly stationary sequences then the covergence is almost sure.

Proof: See Hall and Heyde (1980, p. 36).

0

. Lemma 2.2: A covariance stationary process {X

t} obeys the mean law of large numbers, i.e.,

X n

-1 n

=

n 1: X

t=l t

converges in mean square to a square integrable random variable X. If the process is st~ictly stationary and integrable, then X converges almost surely

n

to an integrable random variable. Proof: See Reve~sz (1968, p. 99).

0

In the following theorem we obtain the ergodic properties of {e t} • Theorem 2.3: Consider {e

t} satisfying the conditions of Theorem 2.1. If

e

<

1, then

r

-1

n a.s. ,

a. s. . -1 n 2r-l

n 1: e - 0

tal t

If

38~

<

1, then for a fixed j ; 0, -1 n

n 1: e e . - - + 0 a.s. tal t t-J

and

a. s. .

and

-1 n

n E

tal 2 e - M

t 2 a.s. (say) ,

where

a. s. ,

Therefore,

Also,

a.s • .

a. s . .

and hence

or

-

a. s. ,

Therefore

M ..

S

(1 -

S

)-1

2 0 1

..

E[e~]

•

-1 n

n E

tal 2

e -_t a.s.

For r ~ 2, we use induction procedure to prove the result. Assume that -1 n

n E

tOIl a. s. ,

for s .. 1, 2,

...

,

I' - 1. We know from Lemma 1 and Lemma 2 that

and

-1 n

n E

tOIl 2r

Note that

Therefore,

2rl r r

cor;

2 .

E[e

t Ft_1] "

e

r _'''0

r

(j) __B et-1J

J 1

r-1

(~)

c:~)

r-j

M ..

_e

r

M

2j +

e

M a.s.

2r _{r j=O} J r 2r

or

MOle

(1-

e

)-1 r;:l

(~)(60)r-j

M 2J

.

2r r r j-O J 61

_ E[e 2r ]

t

Therefore,

a.s.

a. s. .

Note that, if

e

<

1, then

r

a.s. and hence

-1 n 2r-1

n

r

e - - 0 a.s. as n-+ ~. tOIl t

Similarly,

and we g'!t

Consider,

-1 n

n r e

t et -_J. - 0

tOIl a.s. as n - ~.

2 4

- - 6

0 E(et_1) + 61 E(et_1), a.s . ..

E(e~ e~_l)'

Therefore, by Lemma 1, -1 n 2

n

r

e

t

tOIl

2

Similarly,

-1 n 2 2 2 2

n 1.: e e - _{E(e t et_Jo) a.s.} t=l t t-j

as n + ....

In the next section we consider estimation of the parameters a and

§.

3. Estimation of ARCH Models

Given Y

l ' Y2' ... , Yn satisfying (2.U, we desire to estimate ~ and § = (60,61)'. Engle (1982) used the method of scoring to obtain the maximum likelihood estimates, but did not formally derive the asymptotic properties of the maximum likelihood estimators. Engle (1982) indicated that if the conditions of Crowder (1976) are satisfied then the maximum likelihood estimates are asymptotically normal. We derive the limiting distribution of the maximum likelihood estimators by verifying

the conditions of Hall and Heyde (1980, p. 174). We also consider an estimated generalized least squares estimator and derive its limiting distribution.

We consider two particular choices for ~t

(a) ~t is fixed and bounded; and

For case (b), we also assume that ~l a (YO'

Y-l , ••• , Y-P+l) is given and we consider the likelihood given ~l' We will also indicate how the results may be extended for the case where X consists of both fixed and lagged values and

-t

also for the case with X that are not necessarily bounded.

-t

3.1. Maximum Likelihood Estimation

Consider the log likelihood conditional on ~l '

L (y) n

--1

=

-n

~

[in

f (y

IF

l~

where f (y

IF

1) is the conditional density of Y

t given the past and t t

t-X' .. (~', ~' ) . From (2.2),

(3.1)

where

Therefore,

L (y) = constant

+

(2n)-1

~

bih

\+

(2n)-1

~

h-l(Y

t - X a)2

n _ tal \

;I

t=l t -t- (3.2)

The maximum likelihood estimator in of X is the value of X that minimizes Ln(X)' Let X~ a (a~,~') be the true value of X. All probabilities and expectations are taken with respect to the true value XO. We assume that

and

(3.3)

12 so that E[e

t ]

<

m (If Xt is fixed and bounded then we may assume that 8

o S 6

1 S 0.3, so that E[et ]

<

m.) We also assume that ~O is in the interior of a compact set L. Let

(3.4)

+ ~(y - y )'H (y-y )

~ ~o n - ~o

Therefore Xo is as~umed to be in the interior of f. All neighborhoods

defined below will be taken to be contained in

r.

For 0

>

0, and IIx - xoll

<

0, we obtain from the Taylor series expansion that

where

and y* is a point between y and y (not necessarily the same at each occurrence).

o

We include a result from Hall and Heyde (1980) that we use to obtain the asymptotic properties of y •

-n

Theorem 3.1: Suppose that

lim s~ <5-1/T(y)I ..*

<

co _a.s.

,

n-+a> <5 n J.]

1 S i S P + 2

1 S j ~ p + 2 (3.5)

and

a. s. , (3.6)

(3.7)

. where ~ is a positive definite matrix of constants. Then, there exists a sequence of estimators fin} such that in converges to

Yo

almost surely, and for €

>

0

there is an event E with P(E)

>

1 - € and nO such that on E, for n

>

nO'

Y

n satisfies

and L (y)

n - attains a relative minimum at in •

n-~

(aaLynj -

L

N(Q,~) y""yO

If, in addition,

(3.8)

~(- ) L N(O -1 -1)

n

r

_{n -}

rO -

_

'!!

~!!

.

Proof: See Hal! and Heyde (1980, p. 174).

0

We now compute the partial derivatives of Ln(r). Note that,

(y - X a)X' t-1 -t-1- -t-1 '

aL_n -1

-

_aB

..

-2n

1 n

.. -

r

n t=l X't-1-t-1X

(y X )2 X' X

t-! - -t-!~ -t-1-t-!

and

1 n [(y - Xta)

- 2B

-

r

t - - ] (y - X a) [X'X + X' X]

1 _{n t=l h}2 t-l -t-l- -t-t-1 -t-l-t

t

a

2L 2

[(Y

~

2 [2(Y X a)

-h t ]

(y -X a)

n 1 n t

-t-X a)2 t-l -t-l- 4]

a~a~' = -2n t:l1

r

h3 (Yt-l-!t-l~)

t t-1

-t-l-a

2L

1

¥

[(Yt-!t~)2-

ht ]

[~]

n

(y - X a)X

a§a~'

= -

n _t:ll h2 t-l -t-l- -t-l

t 2

[(Yt -1- X a)] 1 n [2(Y -X a) -t -t- ht ]

;t-l:)3 X

r

n t=l h2 (y - -t-1

t t-l

-t-Recall that

1 e

t

=

Zt tv~ where

eO + _{e l} 2 v

t

=

et_l

e ' ,. (eO,e_l) is the true value of ~' = (SO,Sl) and Zt is a sequence of independent N(O,l) variables. Therefore,

and

=

1 nE -~.~ Z X' + _1

e

nE

nt-I v t toot n 1 tal

(Z~-l)

e X'

tool-tool

1 n

- - _{2n tal}E

In the following theorem we verify the conditions (3.5) - (3.8) for the case (a) where we assume that X is fixed and bounded.

-t

Theorem 3.2: Assume that X is fixed and bounded. Also, assume that -t

n-1

¥

X'X __ A ,

tal -t .. t .. as n - "" ,

where ~ is a positive definite matrix. Then, the conditions (3.5) - (3.8) are satisfied with

H - W,. [!!ll

Q].,

Q

!!22 where

,. A(c 2 !!U .. 1 + 2e_l c₃)

r

c 2 c 3 ] !!22 ,.

..

~_{.. c 3 c4}

-1 c

l - E[v2 ] -2 c 2 - E[v2 ]

and

c

=

4

Proof: For a fixed i, let

-~ -1

U =v Z M X . ,

n n n n,1

where M" sup {X .}. i,n n,l Then,

E[U_n

IF

_n-1] = 0 a.s.

and E(U2) is finite. Therefore, by Lemma 2.1,

n

-1 n

n ~ U - 0

t=l t Similarly,

a.s. , as n~ = .

for k

=

0, 1, 2, 3, 4 and

t,s

=

0,1 . Also,

and

Note also that,

=

0 a.s. and

Therefore,

-1 n -1 lim n k v

t et -l X

=

0

n~ t=l -t-l ~

and

Now consider,

a. s. ,

a.s

coV(V~l, v~:j)

=

E[v~lv~:j]

_

{E[v~1]}2

-1 { j-l l l 2 }-l v-I]

:a _{E[v t ] E[ 60} + 6₁ So

l;O

6₁ i~O Zt-l-i - t ~ 6- 3 6j+l E( 2 )

.. 0 1 e_{t -l _j}

Therefore,

as n - "" -1

and since v

t is bounded the convergence is almost sure. Similarly,

-1 n

lim n k

n~ t=l

and

V-IX' X = A c

l

t -t ~t a. s. ,

-1 n -2 -2

lim n k v = E[V

t ] a.s.

n~ t=l t

a. s. ,

and

Since

and

we observe that ~ is positive definite.

Now we establish (3.5). Note that IT

(x)I ..

is a linear combination of terms of

n 1J

the form

ft(y; a, b k)

=

h t -k (Y

t - X a)a(y - X a)b

- ' -t- t-l

-t-l-where a

=

0, 1, 2; b

=

0, 1, 2, 3, 4; and k

=

1, 2, 3. Cons ider for

II

X - Xo

II

<

15 ,

p+2 Ift(y;a, b, k) - f (yo;a, b, k)1 ~ 15 r

- t - i-l

*

where Y

i =

Note that,

A.y. + (l-A.)Y

i 0 and 0 ~

1 1 1 ,

Iy~

- y. 01

~ A.

Iy. - y.

0

1

<

15 •

1 1, 1 1 1,

We will show that af

I

ay: 'X*

<

gt(a,b,k)

,

for all X*

,

lit

_{- xoll}

<

15 ,

-1 n

For example, consider

Note that, Oft

3h~4(Yt

2 6

..

-

- ?St~) _{(Yt - l - X} a) aS

l

-t-l-Now,

Y

t - X-tooa .. et - at where

a .. X (a - a ) •

t -t - -0

Then,

8 -1 n 2 6

Note that, since we assumed that E[e

t ] is finite, n _t=lL etet_l

,

-1 n 2

and -1 n 6 surely finite Similarly,

n L e

t n L et-1 converge almost to constants.

t-1 t=l

other terms of

IT (x)I ..

can be bounded to establish (3.5). n 1.J

Now, using Scott's martingale central limit theorem (see Scott (1973» we 1

(aL~

will show that n~ ~ is asymptotically normal. aX y"y

- -0 Define,

x-xo

where

D'

=

(Db,

11

It is clear that {S ,F } is a martingale. Let n n

V2 _ E[S2/ F ]

n n n-l

and

2 _ E[S2]

s

.

n n

Then,

V2

e

2 n -2 2 n -1 n'X'X n

- 2 E vt e t - l n'X' X !)O + E v

n 1 _t-l -O-t-l-t-l _t-l t -O-t-t-O

n _{-2 2 2}

+~ E vt (n_l + n₂e_t__l) t=l

and

2 2 n n

s - 26 c E n' X' X n + c n' E X' X nO n 1 3 tal -O-t-l-t-l -0 1 -0 tl tt

-Note that,

and

P

- 1 as n - co

o -1 2 ,

~im n s -

D~

D .

n-- n

Since we assumed that

E[e~]

<

co, it follows that

Elv-

3/ 2 z3₁

<

co

t t '

and the Lindeberg condition is satisfied. Therefore,

-1 L

s S - N(O,l) n n

o

Now we consider the case ~t = (Y

t-l , Yt-2, .•• , Yt-p), We assume that the

roots of the characteristic equation

P p-l

m - a m₁ - - a = 0

p

lie inside the unit circle. Then, Y

t can be written as an infinite moving

average as,

co

1: w.e .

j=O J t-J where {w.l satisfy

J

(3.10)

=

1

= 0

, j • 0

, j

<

0

-Theorem 3.3:

In the following theorem we obtain the asymptotic properties of the maximum

likelihood estimator.

Assume that X

=

(y l' Y 2' •.. , Y ) and that the roots of

-t t- t- t-p

the equation (3.9) lie inside the unit circle. Then the conditions (3.5) - (3.8)

are satisfied with

G

-~l

2)

H = W=

- 0 !!22

where

and ~22 is as defined in Theorem 3.2. Proof: For a fixed i, let

U

=

v-1e Y

Then,

and E[U2] is finite. Therefore,

n

-1 n

n I: U - 0 a.s. t=l t

"Also,

-1 n -2 2

n I: v (e -v ) e lX 1 .

t=l t t t t- t- ,4

and

- 0 a.s. ,

as n + 00

Using the arguments similar to those in Theorem 3.2, we get

= 0 a. s. ,

= !!22 a.s . .

Also,

r

=y

-0

=

tim n+OO

-1 n n I: t=l

262 -1 n -2e2 X' X

+ tim n I: v

n+OO 1 t=l t t-l-t-l-t-l Consider, for fixed i and j

-1 n -1 00 00

-1 n -1

n I: v Y .Y . = I: I: w

k w n I: vt e t - i - k e

t=l t t-4 t-J k=O s=o s t=l t-j-s

Now for fixed q and r, consider

-1 n n I: t=l

-1

v e e

I f q=r=l, then

-1 n -1 2 -1 n -1

n 1: _{v t e t - 1}

=

(n6

1) 1: vt (vt-60)

tOIl tOIl

-1 2 -E[v

t et_1J a.s.

.If q"r>l, then it can be shown that

for j > q. Therefore,

a.s.

-1

Now if q ~ r, then v e e are uncorrelated and hence t t-q t-r

-1 e e .. E[V

t et et J

t-q t-r -q -r

.. 0 a.s. Using Lemma 6.3.1 of Fuller (1976), we get

-1 n -1 -1

lim n 1: v Y .Y . " E[V

t Yt .Yt.J a.s . .

n~ tOIl t t-1 t-J -1 -J

Similarly,

-1 n -2 -2

lim n 1: v Y Y . " E[V

t Yt 'Yt .J a.s. ,

n~ tOIl t t-i t-J -1 -J

and hence

I f

for any arbitrary ~O • -1

vp+1(Yp ' Yp- l ' .... Yl )·

-1 , Now. E[v

t !t!t

1

is the variance covariance matrix of

then

or

~o !~l • 0 a.s . .

However. we know that the variance covariance matrix of !p+l = (Y_p.Y_p-_l•..•• Y₁) is positive definite. Therefore.

Now to establish (3.5). note of the form

~tl is positive definite. that IT (Y)l

i , is a linear

-n - J combination of terms

-k a X a)b ql q2 q3 q4

ft(v;a,b.k.a) • h t (Yt - Xta) (y 1 - 1 X 1 i X l 'X X 0

~ ~ - - t- -t- - t - , t-.J t.r t.~

where qi

=

0 or 1 and a. band k range from 0 to 4. Again, for example. consider

f

t(y_;2.4.3.0_)

=

h-3_{(y - X a)2(y - X a)4} t t -t- t-1

-t-1-Then

2 6 ₀₆

p2

t

6 2 02

t

y2 . 6 :ii constant [e_{t e t - 1} + Y 1 .e + e

t_1 i=l t- -1 t i=l t-1 + 08 p2

~

t

y2 y2 . ]

3

a_t-1) y_t-1.-. 11

·Since

E[e~2]

is assumed to be finite, we get lim sup 0-1

IT

(x*)I ..

<

~

a.s.

n~ 0+0 -n l.J

Now we verify (3.8).

S

=

n'

(aL

n\ n _ aX ')

Let

where

n'

= (~~,

n

1,

n

2) is an arbitrary vector of constants such that

n'n ;

° .

Then,

n _!.<

r

v"Z n'X' t=l t t_O-t

v

2

=

E[S2

IF

1]

n n

n-82 ~ -2 2

= 2 w v e n'X' X n +

1 t=l t t-l.0·t-l·t-l.0

and

-1

v n'X'X n t .O-t·t.O

1 , as n - 00

Since we assumed

E[el~J

<

00 ,

t

and

Therefore, by Scott's martingale central limit theorem,

-1 L

s S - N(O,l) , n n

and hence

L -1

- YO) - N(Q,~ )

o

likelihood estimator is still consistent and asymptotically normal. It is easy to see that if ~t

=

(l,Y

t- l , Yt - 2 , ... , Yt-p+l) the maximum

I f X -t

is fixed but not necessarily bounded then ~ may converge to a

O at a rate faster

-~ -1

than n . For example, if X

t = t, then (a - aO) is 0p(n ).

Now we consider the least squares estimation of

y.

3.2. Least Squares Estimation:

The maximum likelihood estimates considered in 3.1 do not have explicit expressions and are estimated using iterative procedures. We now consider the ordinary and estimated generalized least squares estimates of

y.

The least squares estimates are obtained as follows:

Step 1: of ~.

Regress Y on X to obtain the ordinary least squares estimator a

-Step 2: Regress -2e

t on a

-

-

-2

v

t

..

60 + 61 e t - l

.

Step 3: Regress v--1 -2e

t on t

-2

-column of ones and e

t-l to get 60 and 61 , Let

--1 --1 -2

v

t and vt et-l to get an estimated generalized

least squares estimates

eO

and

e

l • Step 4: Regress

v-~

Y

t on

v-~

X to get

- t t -t

estimate

a

of a

-

-Let v

t ..

eO

an estimated

A -2

+ 6 let _l

generalized least squares

We now study the properties of ~, ~, ~ and~. We first consider the case where X is fixed and bounded.

-t

Theorem 3.4: Let {!t} and {Y

t} satisfy the conditions of Theorem 3.2. Let

YO be in the interior of f. Then

~ - L

n

(r -

YO) - N(2, ~O)

and where

y'"

(~',§') B ..

o

B .. -1 [ 2 -1 cr A

o

o

--1

-1 2 !!22

o

]

--1

~l !!22

A .. E

-1

and c

l and !!22 as defined in Theorem 3.2.

Proof: Note that

We know that

Consider,

S ..

n'

X'e

n _0

...

,

x' )

-n

where

=

n1: _{bte t}

tOIl

b ..

~

n.

0 X .

t i-l~' t,~

and ~O is an arbitrary vector of constants with

nb

nO ~ O.

a martingale with,

v

2 .. E[S2

IF

1]

n n

n-Note that {S ,F } is n n

and

2 2

s '" (J

n

Therefore,

-2 2

s V n n

and

L

1

-1 2

n s_n - _!lO, (J2~!lO

Since E[e:l is finite, the Lindeberg condition is satisfied and

Therefore,

~ - L 2 -1

n (':-':O)-N(Q,(J~ ).

Now consider,

_ =[

(n-1)

~ n -2

E e

t=2 t-1

where

n E t=2

n E t=2

-e = Y

t - Xa

t

-t-and

n -1 n -2

r

_{et-la t - l} + n

r

a

t=2 t=2 t-l

Similarly, -1 n

n

r

t=2

and

-4

e

=

t-l

-1 n -2 -2

n

r

e e

t=2 t-l t

-1 n 2 2 0 (n-~)

=

n t _{et_le t} +

t=2 P

Therefore,

-1 n 2

r

[n-~

n

1 n

r

e

r

dt ]

~

-

t-2 t ta ₂

o

(n-~)

n (a-a) -

- -

+

[ -1

n

2 -1 n 4 _-~ n p

n E e n

r

e n

r

dte~_l

t-2 t t=2 t t=2

where

d

t - (Z2 -t l)vt

Note that,

-1 n 2 n

r

e

t=2 t

-1 n 4 n

r

e

t=2 t as n + 00 •

Con!3 ider ,

r

- A_-I a. s. ,

where

n

and

Using the usual arguments we get

-1

L

s S - N(O,1) , n n

and hence

~ - L - 1 - 1

n (~-~)--+ N(Q, ~l ~l ~l ) • Z

Note that (Zt-l) and e

t are uncorrelated and hence ~ and

e

are asymptotically independent. Now we consider ~ obtained by step 3.

Using arguments as above, we get

n _{- -2} n _--2

-2 ]

-1

[n

--1 -2 ]

[ti

2 vt 1: v

e t - l 1: v e

e

₌ taZ t t=Z t t

-

n _{--Z -4 n --1 -Z -2}

--2 -2 _{1: v}

e t - l 1: v e e

t-Z vt e t - l t=Z t t=Z t t-l t and -1 !!Z2 n 1: t=2 n 1: t=Z

+ 0 (1) • p

Consider,

Then, S is a martingale with n

v

2 .. Z n1: (n Z Z

1 + nZet_1) a. s. ,

n _t=Z and Z Z(n-l) E(n l + 2 Z

s = n

Since

E[e~]

is finite, Sn satisfies the Lindeberg condition and

-1 L

s S - N(O,l)

n n

Therefore,

A-l A-l

Now we consider the regression of v

t Yt on vt X-t

where

Let

Then, as n -+ ... ,

and

Consider,

Note that,

=

¥ (

~

n X

.)v·~

Z t=l i=l i,O t,1 t t

Note again that S is a martingale with

n

and

s-IS

~N(O,l)

n n

and

2 A

Since (Z -1) and Z are uncorrelated a and 8 are asymptotically independent.

0

t t . . . .

Note that

and the equality hold only.if 8

1 = O. Therefore, the maximum likelihood estimator is asymptotically the best among the three estimators considered and the estimated

generalized least squares estimator is asymptotically better than the ordinary

least squares estimator.

Here we have assumed that X is fixed and bounded. Suppose X is fixed

.. t .. t

and satisfies

n lim

r

n - tal

_ co

- a -h,i,j

and

G

n 2

~

-1 2

lim

r

X . X . = 0 ,

n - tal t,1 n,1

for i = 1, 2, ..• , p; j - 1, 2, ••• , p , where

o

=

diag{(~

x

2.\

..n

\(=1

t~

Then, it can be shown that

and

Now we obtain the asymptotic properties of the least squares estimators for the case when ~t

=

_{(Y t - 1 • Yt-2' ...• Yt - p )·}

Theorem 3.5: Assume that X saisfies the conditions in Theorem 3.3.

~t Then.

and

where

B ,.

[9

~3

o

~

Q 2

9-1

1 _{+ 6 Q-1 Q Q-1}

~

=

a 1~1 ~2~1

for j ~ i.

2 (Q2)·· = E[Y_t .Y_{t .e t 1]}

~ 1J -1 -J

-CD k+i-l

=

y 2(0) 1: wkw_k+· _{.6 1}

and

-1

=

E[v Y

t 'Yt

.J

t -1. -]

~ -1 2

= k~O _{wk wk}+_{i - 1 E[vt et_i_kJ .}

Proof: Note that, -1 n

n E Y .Y .

tal t-1. t-J

and hence

a.s.

Consider,

S a n'X'e

n -0- ...

n

=

E b e t=l t t where

b a t

n.

OY . t i=l 1., t-1.

a.s.

and n_

O is an arbitrary vector with n'n-0- ~ O. Then Sn is a martingale with,

v

2 = 6 n X' 2

0 n'

x'x

nO + 61 E n' X e

t _

1 ~O a.s.

,

n -0 _{- t=l -0 -t -t}

and

2 _n'

gl~O + n 6₁ n'

E[~~~te~_lJ

s = n 6 _~O

n 0 -0 -0

Note that, for j ~ i,

-1 n 2

n E Y .Y e

t=l t-1. t-j t-l

=

~ ~ -1 n 2

E E _{wkws nEe} . ke .e 1

t-Therefore,

and

-1 2 n s_n

-Since E[e:J is finite, the Lindeberg condition is satisfied and

-1 L

s S - N ( O , I ) . n n

Therefore,

Using the arguments similar to those of Theorem 3.4, it follows that

n~(~-~)~(2'

2

~~l~l~~l)

n~(~-~)~(2' 2~;~ ~l~;~)

,

and a and ~ are asymptotically independent.

A

Now, to obtain the limiting distribution of ~, consider

=

a. s.,

where gn and ~O are as defined in Theorem 3.4.

v2

= '

X'G-1Xn n nO - -n --0 and

From Theorem 3.3, we know that

-1 n -1 -1

n

r

v

t Yt .Yt . - - E[vt Yt .Y~.J a.s.

t=l -L -J -L ~-J

Therefore,

. and

-2 2 s V

n n

L l

-1 2

n'

E[v-1X'X J

n sn - - -0 t -t-t ~O·

Using Scott's martingale central limit theorem, we get

-1

L

s S -N(O,l)

n n

and

Note also that ~ and ~ are asymptotically independent.

0

If X involves both fixed and lagged variables then one can obtain results -t

si.milar to those of Fuller, Hasza and Goebel(l981). Also, if Y_{t process has a unit}

root, we can obtain the asymptotic distribution of the least squares estimator. Consider, for example,

and {e

t} satisfies the conditions of Theorem 3.4. The least squares estimator of a

l is given by,

a

l

=

[t~2

Y;-l]

-1

Then,

2 n 2 -1 n

=

[n - r Y J [n-l r Y e J

If 6

1= 0, Dickey and Fuller (1979) obtained the asymptotic distribution of n(a

1 - 1). We now show that even if 61 ~ 0, n(al - 1) has the same limiting distribution.

We know that

-1 n 2 .tim n

r

e n-- t=2 t

Now consider,

T = n-~ Y

n-1 n

n-l

a. z* =

r

i=l 1.,n i,n and

r

,. n-2 n

r

y2

n _t=2 t-l

a.s.

where

and

z*

=

(Z*l ' ..• ,Z* 1 ) ,

,n n- ,n

= M e

-n "'n

mit(n)

=

(i,t) - th element of ~n

-!.. 1

=

2(2n-1) 2 Cos[4n-2)- (2t-l)(2i-1)'lf]

Using Scott's martingale central theorem, it follows that, for any fixed k,

where !k is an k x k identity matrix. Now using the arguments similar to

Hasza (1977) and Pantula (1982) it follows that n(~l - 1) has the same limiting distribution as that obtained by Dickey and Fuller (1979). Similarly, the results for pth order ARCH models may be obtained.

4. Summary:

We have considered linear regression models with autoregressive conditionally heteroscedastic errors, introduced by Engel (1982). We have obtained a series representation for the first order ARCH errors. We hav.e used the representation to derive the ergodic properties of the errors. Similar representation can be ob-tained for the qth (q> 1) order ARCH errors but are not presented here. A special case where the conditional error variance is of the form 8

0 + 81

t

a.e 2

.

,

where j=l J t-J

-1 -1

[q+l-j] will be considered elsewhere. a. = q or a. = 2[q(q+1)]

J J

We have considered the maximum likelihood estimation of ARCH regression models. The maximum likelihood estimators do not have explicit algebraic form and are computed using iterative methods. We have shown that the maximum likelihood estimators are strongly consistent and asymptotically normal. We have also shown that the least squares estimator and an estimated generalized least squares estimator are asymptotically normal. For a random walk model

(Y

t = alYt-l + et,al = 1) with ARCH errors, we have shown that the asymptotic distribution of the least squares estimator of a

l is the distribution obtained by Dickey and Fuller (1979) for the homoscedastic case.

Acknowledgements

BIBLIOGRAPHY

Chung, K. L. (1974). A Course in Probability Theory. Academic Press, NY. Crowder, M. J. (1976). Maximum Likelihood Estimation for Dependent

Observations, Journal of the Royal Statistical Society, Series B, 45-53 • . Dickey, D. A. and W. A. Fuller (1979). Distribution of the Estimators for

Autoregressive Time Series with a Unit Root, Journal of American Statistical Association, Z~, 427-531.

Engle, R. F. (1982). Autoregressive Conditional Heteroscedasticity with Estimates of United Kingdom Inflation, Econometrica,

2Q,

987-1007. Fuller, W. A. (1976). Introduction to Statistical Time Series. Wiley, NY. Fuller, W. A., D. P. Hasza, and J. J. Goebel (1981). Estimation of the

Parameters of Stochastic Difference Equations, The Annals of Statistics,

2,

531-543.

Granger, C. W. J. and A. Anderson (1978). An Introduction to Bilinear Time-Series Models. Vandenhoeck and Ruprecht, Gtlttingen.

Hall, P. and C. C. Heyde (1980). Martingale Limit Theory and Its Application. Academic Press, NY.

Hasza, D. P. (1977). Estimation in Nonstationary Time Series. Unpublished Ph.D. Thesis, Iowa State University, Ames, Iowa.

Jones, R. H. (1965). An Experiment in Nonlinear Prediction, Journal of Applied Meterology, ~, 701-705.

Pantula, S. G. (1982). Properties of Estimator of the Parameters of Autoregressive Time Series. Unpublished Ph.D. Thesis, Iowa State University, Ames, Iowa. Priestly, M. B. (1978). Nonlinear Models in Time Series Analysis,

Revesz, P. (1968). Scott, D. J. (1973).

The Laws of Large Numbers. Academic Press, NY.