If is continuous and if then is a surjection.

By this property, is a surjection for a large class of For example, let be free of and let the growth rate of be not faster than linear as Then with satisfying A2.2.1 we have as for all Hence, A2.6.1 holds. In the case where the growth rate of is faster than linear

as and for some we also

have that as for all and A2.6.1 holds.

In what follows by stability of a set for we mean it in the Lyapunov sense, i.e., a nonnegative continuously differentiable function

exists such that and

for some where

Theorem 2.6.2 Assume A2.2.1, A2.2.2, and A2.6.2 hold, and that

is continuous and for a given A2.6.1 holds. If defined by (2.1.1)– (2.1.3) with any initial value converges to a limit independent of

then belongs to the unique stable set of

Proof. Since by A2.2.2 and by conti-

nuity of exists with such that

Hence, By continuity of J is closed, and hence by A2.2.2,

Since we must have Denote by the connected subset of containing The minimizer set of that contains is closed and is contained in Since is a connected set and by A2.2.2 is nowhere dense, is a constant.

By continuity of all connected root-sets are closed and they are separated. Thus, there exists a such that

i.e., contains no root of other than those located in Set

Then and

Therefore, by definition, is stable for

We have to show that and is the unique stable root-set. Let be the connected set of such that contains By continuity of for an arbitrary small

exist such that and the distance between the interval and the set is positive; i.e.,

We first show that, for any and there exist

and such that, for any if then

By Theorem 2.2.1, for with sufficiently large there will be no truncation for (2.1.1)–(2.1.3), and

For any let By A2.6.2, sufficiently small and large enough exist such that for any

If for then (2.6.10) immediately

follows by setting Assume for some

Let be the first such one. Then

By (2.6.11), however,

which contradicts (2.6.12). Thus and (2.6.10)

is verified.

For a given we now prove the existence of such that for any if where the dependence of

on and on the initial value is emphasized. For simplicity of writing, is written as in the sequel.

Assume the assertion is not true; i.e., for any exists such that

and for some

If there exists an with then with exists because is connected and with

This yields a contradictory inequality:

where the first inequality follows from A2.2.2 while the second inequality is because is the minimizer of

Consequently, for any and

and a subsequence of exists, also denoted by for

notational simplicity, such that By the continuity

of

Hence, by the fact

By (2.6.10) and the fact we can choose sufficiently small T and large enough N such that

and i.e.,

for any By (2.6.10), exists with the

property such that

Because as for sufficiently large N, by (2.6.10) the last term of (2.6.15) is Then

By (2.6.10) and the continuity of the third term on the right hand side of (2.6.16) is and by A2.6.2 (Since

with for all sufficiently large N.), the norm of the second term on the right-hand side of (2.6.16) is also as Hence by A2.2.2 and (2.6.13), some exists such that the right-hand side of (2.6.16) is less than for all sufficiently large N if T is small

enough. By noticing and mentioned

above, from (2.6.14) it follows that the left-hand side of (2.6.16) tends to a nonnegative limit as The obtained contradiction shows

that exists such that for any if

With _{fixed for any} by A2.6.1 exists such that

By and the arbitrary smallness of from this it

follows that Since by assumption, we have

which means that is stable. If another stable set existed such that then by the same argument would belong to The contradiction shows that the uniqueness of the stable set.

2.7.

Robustness of Stochastic Approximation

Algorithms

In this section for the single root case, i.e, the case we consider the behavior of SA algorithms when conditions for convergence of algorithms to are not exactly satisfied. It will be shown that a “small” violation of conditions will cause no big effect on the behavior of the algorithm.

The following result known as Kronecker lemma will be used several times in the sequel. We state it separately for convenience of reference.

Kronecker Lemma. If where is a sequence of positive numbers nondecreasingly diverging to infinity and is a sequence of matrices, then

Proof. Set Since

follows that

as and then

We still consider the algorithm given by (2.1.1)–(2.1.3), where de- notes the estimate for at time but may not be the exact root of As a matter of fact, the following set of conditions will be used to replace A2.2.1–A2.2.4:

A2.7.1 nonincreasingly tends to zero, and exists such that

A2.7.2 There exists a nonnegative twice continuously differentiable func-

tion such that and

A2.7.3 For sample path the observation noise satisfies the fol- lowing condition

A2.7.4 is continuous, but is not necessary to be the root of

Comparing A2.7.1–A2.7.4 with A2.2.1–A2.2.4, we see the following conditions required here are not assumed in Section 2.2: nonincreasing

property of condition (2.7.1), nonnegativity of divergence of to infinity and continuity of but in (2.7.2), in (2.7.3), and

are allowed to be greater than zero.

Concerning we note that from the convergence of

it follows that i) A2.2.3 holds and ii) by the Kronecker lemma because is nonincreasing. We will demonstrate how does the deviation from of the estimate given by (2.1.1)–(2.1.3) depend on and

For used in (2.1.1) define Since as

can be taken sufficiently large such that

Let the initial truncation bound used in (2.1.1) and (2.1.2) be large enough such that

Take real numbers such that

Since is continuous, an exists such that

Denote

and

where denotes the matrix consisting of the second partial deriva- tives of

Since we have for any and hence

Set

We will only consider those in (2.7.2) for which where is given in (2.7.7). From (2.7.7) and (2.7.8) it is seen that

Consequently, by (2.7.2), a given by (2.7.12) is positive.

By continuity of and and exist

such that the following inequalities hold:

By A2.7.3 for can be taken sufficiently large such that

Lemma 2.7.1 Assume A2.7.1, A2.7.2, A2.7.4 hold with given in (2.7.3) being less than or equal to If for given by (2.1.1)– (2.1.3) with (2.7.5) fulfilled, for some where K is given in (2.7.18), then for any

Proof. Because is nondecreasing as _{T increases, it suffices to} prove _{the lemma for}

Then for any we have

and hence

which incorporating with the definition of leads to

On the other hand, from (2.7.20) and (2.7.21) it follows that

From (2.7.9) we have

By a partial summation we have

Applying (2.7.3) to the first two terms on the right-hand side of (2.7.25), and (2.7.1) and (2.7.3) to the last term we find