A Trust Region Algorithm Using Curve-Linear Searching Direction for Unconstrained Optimization

(1)

A Trust Region Algorithm Using Curve-Linear

Searching Direction for Unconstrained

Optimization

Shu-ping Yang

School of Mathematical Science and Computing Technology, Central South University, Changsha, 410083 [email protected]

Xiu-gui Yuan, Zai-ming Liu

School of Mathematical Science and Computing Technology, Central South University, Changsha, 410083 [email protected]

Abstract—In the paper, aimed at the shortcoming of trust

region method, we proposed a algorithm using negative curvature direction as its searching direction. The convergence of the algorithm was given. Furthermore, combing trust region method and curve-linear searching techniques, a trust region algorithm, using general curve-linear searching direction, was proposed. We proved its efficiency and feasibility. The algorithm has adjustability and can select or update its searching direction according to the iteration. This allows the algorithm that has the properties of curve-linear searching method and the global convergence of trust region method. Finally, we indicate that some searching directions of common methods can be as a special searching direction of the general method.

Index Terms—nonlinear programming, unconstrained

optimization, trust region method, curve-linear searching method, searching direction, quadratic model, directions of negative curvature

I. UNCONSTRAINED OPTIMIZATION PROBLEMS[1] In many problems of unconstrained optimization such as :

(P1)

min ( )

n

x R

f x

∈

Its solution has been appealed to many peoples to do it. People created many algorithms aiming at question (P1) and presented some methods as the trust region method; Newton method; DFP method and BFGS method and so on. At these methods, people using straight line as its searching direction to searching line in general. with these methods , they have some defect as “saw tooth phenomenon” and “ local convergence”, some people presented the trust region method and curve-linear searching techniques to quality the global convergence of iterative method Curve-linear searching method is a way of searching direction down to a curve, which will avoid certain defects compared to rectilinear direction searching[2]. In trust region method, we gain new iteration step length based on a partial model of minimized objective function on a constrained ellipsoid domain centered at the current iteration point, and the

diameter of the ellipsoid is determined by the pattern of anticipation objective function of the model. Researches have been done by M. J. D. Powell [3], J. E. Dennis jr. and H. H. W. Mei[4], J. J. More [5], D. C. Sorensen[6, 7], D .C. Sorensen and J. J. More [8], G. A. Shulty. etc[9] and Yuan Y. etc[10-13]. It is needed to calculating

one of the partial models of

f x

( )

at the point of

x

k_:

1 ( )

2

T T

k k k k

Q w

=

f

+

g w

+

w G w

It is solved by solving its equivalence problems: It equals to:

(P2)

min

{

Q

_k

( )

w

;

w

≤ ∆

_k

}

(P3)

λ

≥

0

s.t.

G

_k

+

λ

I

is a positive semi-definite matrix,

and

(

)

( )

k

G

I w

g

w

λ

+

= −

⎧⎪

⎨

_{≤ ∆}

⎪⎩

In many studies, it is generally assumed

that

G

k

+

λ

I

_{is a positive definite matrix, so the problem}

occurs when

G

k _{is not a positive semi-definite matrix}

and is possibly a local minimum point. If

g

k

=

0

_{, we can}

gain zero solution to question(P3)exclusively, and the

iterative procedure discontinues. When

G

k _{is not a}

positive semi-definite matrix, More & Sorensen[8], which means situations hard to cope with appears,

indicating that

G

k _{has one negative eigenvalue}

λ

1 _, where

g

k_{is orthogonal to the null space of}

(

G

k

−

λ

1

I

)

,

and

(

k 1

)

k k

G

−

λ

I

+

g

< ∆

(2)

largest negative eigen value. The solution is that the iterative step length be selected as:

(

1

)

k k k k k

p

= −

G

−

λ

I

+

g

+

ξ ν

, where

ν

_k is eigenvector of

g

_k relative to

λ

₁ .Select

ξ

_k s.t

p

_k

= ∆

_k

These methods above only involve single directions, so that if problems occur, the algorithm discontinues. Aiming at these problems, we present corresponding improved methods, using a curve-linear searching direction consisting of two descent directions.

II. IMPROVEMENT OF TRUST REGION ALGORITHMS

Algorithm 1

(1) Where

0 < ≤ <

µ η

1, 0

<

γ

1

< <

1 γ

2

,

x

0

,

∆

0 (2) Given that

x

k

,

∆

k_{, calculate}

g

k

,

G

k

(3) Decompose symmetric matrix

G

k_{by Bunch-parlett}

decomposition method [13]

T k k k

k

L

D

L

G

=

Where

L

k _{is a triangular identity matrix}

(

( ) ( )

)

1

,...

k n k

k

diag

d

D

=

(4) (a) If

g

k

=

0

_and

d

(

i

n

)

k

i

0

1 ,

2 ...

)

(

≥

=

,

let

x

∗

=

x

k_{, then stop. If not, go to the}

Step(b).

(b) If

G

k_{is not a positive semi-definite matrix, and}

g

k_is

orthogonal to the null space of

( ) 1

k k

G

−

λ

I

, where ) ( 1

k

λ

the largest negative eigenvalue, then go to the Step(c). If not, switch to Step (5).

(c) Use Fletcher-Freeman[14]method to determine

negative curvature direction

d

k_。

(d) Let

x

k+1

=

x

( )

α

k

=

x

k

+

α

k

d

k_,

α

k_{is determined}

by the conditions below:

(

)

(

)

[

k

]

T k k k T k k

T k

k

d

G

d

g

d

x

f

+

≥

+

∇

α

η

α

(

)

( )

1

2

T T

k k k k k k k k

f x

+

α

d

≤

f x

+

µα

g d

+

µα

d G d

k

:

= +

k

1;

switch to Step (2) (5)

i) Establish a secondary model:

( )

w

f

g

w

G

w

Q

_k _k _kT T _k

2

1 +

+

=

ii) Find the solution of equation below:

( )

{

}

arg min

;

k k k

p

=

Q

w

≤ ∆

iii) Calculate

k k k

pred

ared

=

ρ

.

If

ρ

k

<

µ

_{, then}

∆ = ∆

k

:

γ

1 k_{and switch to Step ( );}

if

ρ

k

≥

µ

_{, then}

x

k+1

=

x

k

+

p

k

,

k

:

=

k

+

1

_;

if

ρ

k

>

η

_{, then}

∆

k+1

:

=

γ

2

∆

k_{, or else}

∆

k+1

:

=

∆

k_and

switch to Step (4). Theorem 1

If

f

R

n

→

:

, where

n

R

D

x

₀

∈

⊂

and

L x

( )

0

=

{

x

∈

D f x

;

( )

≤

f x

( )

0

}

is a compact subset of

D

.There is

( )

0

;

k k

x

∈

L x

G

≤

M

,

{ }

x

k _{is a iteration point}

sequence based on Algorithm 1. Then

lim

T

0 ,

lim

T

0

k k k k k

k→∞

g d

=

k→∞

d G d

=

₍₁₎

Prove:

If

{

}

( )

1

;

0,

Null(

1

)

k

k k k k

D

=

x g

=

g

⊥

G

−

λ

I

and ( )

1

Null(

G

_k

−

λ

k

I

)

is the null space of

( ) 1

k k

G

−

λ

I

.

Let

D

2

=

{ }

x

k

−

D

1_{, and}

{ }

x

k _{is an infinite point set.}

That if

{ }

x

k _{is finite, then based on Algorithm 1,}

(

k k

)

k

w

G

x

w

ared

=

−

+

θ

2

1

,

1 ,

2

T T

k k k k k k

pred

=

p w

−

w G w

0,

0

k k

g

=

d

=

Then conflict arises. At least one of

D

1_and

D

2_is infinite.

If

D

1_{is infinite,}

L x

( )

0 _{is a compact set, then}

{ }

x

k

has at least one accumulation point. Let

x

k

→

x

∗_{, based}

on algorithm 1:,

⎥⎦

⎤

⎢⎣

⎡

₊

−

≥

−

₊ T _k _k

k k k

T k k k

k

f

g

d

G

d

f

₁ 2

2

1 _α

α

µ

and

d

_kT

G

_k

d

_k

≤

0 ,

d

_kT

g

_k

≤

0 ( )

x

f

∵

is continuous function, then

(

→

∞

)

→

−

f

₊

k

f

_k _k ₁

0 ,

so that

2

lim

_k _kT _k _k

0, lim

_k _kT _k

0

k→∞

α

d G d

=

k→∞

α

g d

=

Let

lim

_k

0

k→∞

α

=

, based on the continuity of

(

x

k

d

k

)

f

+

α

∇

and algorithm 1, step (d):

(

)

(

)

1 lim

1

T

k k k k

T T

k

k k k k k k

f x

d

g d

d G d

α

η

α

→∞

∇

+

=

≤ <

(3)

Then conflict arises. So

α

_k

≥

ε

,

(

k

≥

k

₁

)

and

lim

_kT _k _k

0, lim

T_k _k

0

k→∞

d G d

=

k→∞

g d

=

.

If

D

2 is infinite, based on Algorithm 2:

x

k

∈

D

2 is drawn by trust region algorithm, and based on the convergence of the algorithm, obviously

lim

_kT _k _k

0

k→∞

d G d

=

is supported. Then (1) is proved.

Based on the conclusion of Theorem 1 the theorem below can be proved:

Theorem 2 Under conditions of Theorem 1, and

{ }

d

k is limited, then

∃ >

c

0

, s.t.

(

→

∗

)

→

≥

c

g

x

_∗

k

d

_k _k

,

_k

,

Then

( )

=

0 ∇

f

x

_∗ and

G

( )

x

_∗ is positive semi-definite.

III. GENERAL SEARCHING METHODS

To question (P1),

let

x

( )

α

=

x

+

φ

1

( )

α

s

+

φ

2

( )

α

d

, where

( )

α

φ

₁ and

φ

₂

( )

α

are continuous functions under condition that

α β

∈

(where

β

is a compact set in

R

).Given that

φ

₁

( )

0 =

φ

₂

( )

0 =

0

(equals to

x

( )

α

=

x

);

s,

d

are descent directions of

f

( )

x

at the point of

x

.Create a secondary model of

f

( )

x

at the point of

x

:

( )

p

f

( )

x

g

p

Gp

Q

T T

2

1 +

+

=

(2)

Where

p

( ) ( )

α

=

x

α

−

x

=

φ

1

( )

α

s

+

φ

2

( )

α

d

(3) To guarantee that

Q

( )

p

≤

f

( )

x

, then

( )

α

β

φ

2

≥

0 ,

∈

φ

1

( )

α

≥

0

(4) s.t. Algorithm 1 is modified. To make it further, we gain

a general trust region algorithm using curve-linear searching direction for unconstrained optimization: Algorithm 2

Step1-3 are the same as (1)-(3) in Algorithm 1. Step4 is same as (4) (a)-(c) in Algorithm 1. (d) Let

x

_k₊₁

=

x

( )

α

_k

=

x

_k

+

p

( )

α

_k .

k

α

is determined by the conditions below:

( )

(

)

(

)

T

T T

k k k k k k k k

f x

p

α

d

η α

⎡

d G d

g d

⎤

∇

+

≥

_⎣

+

_⎦

( )

(

)

( )

1

2

k k

T T

k k k k k k

f x

p

f x

g d

d G d

α

µα

+

≤

+

,

:

1 k

= +

k

; switch to Step2.

Definition 1

f

0

( )

x

is defined as generalized derivative

of

f

( )

x

at the point of

x

,

if 0

( )

(

)

( )

0

lim sup

t

f x t

f x

f

x

t

→ +

+ −

=

.

Definition 2

ζ

is defined as generalized successive derivative of

f

( )

x

at the point of

x

, if

( )

x

≥

ζ

f

0 .Note generalized successive derivative of

f

( )

x

at the point of

x

as

α

_β

f

( )

x

.That is

( )

{

ζ

( )

ζ

}

α

_β

f

x

=

;

f

0

x

≥

.

Create the secondary model as below:

( )

(

( )

)

( )

(

( )

(

)

( )

(

)

1 2

2

1 2

2

1 2

0

1

0

2

1

0

2

0

T T

T

F

f x

g s

g d

g s

g d

s

d

G

s

d

α

µ

α

ν

α

µ

=

+

⎡

⎤

+

_⎣

+

_⎦

⎡

⎤

+

_⎣

+

_⎦

⎡

⎤

×

_⎣

+

_⎦

(5)

Obvious (5) is a generalized pattern of (2). To satisfy that

F

( )

α

≤

f

( )

x

, then

( )

0 0,

( )

0 0,

(

1, 2

)

i i

i

µ

≥

ν

≥

=

Let

( )

0

( ) ( )

0

( ) (

)

,

1, 2

i i i i

i

µ α

=

φ α ν α

=

ν α

=

, then:

( )

1

( )

1

0

0 lim

t

φ

µ

→ +

=

, ₂ 2

( )

0

lim

t

φ

µ

→ +

=

(

i

=

1, 2 ,

)

ζ φ α

∈

( )

Theorem 3

Assume that

f

( )

x

is a second order continuous differentiable function on

R

n ,

φ

₁

( ) ( )

α

,

φ

₂

α

are positive continuous functions on

β

(a compact subset on

R

)

→

R

,

g

( )

α

∈

α

_β

f x

(

( )

α

)

is continuous from the right of point

x

.If:

( ) ( )

0 ,

1 ,

0 ≥

0 ,

( )

0 ≥

0 ,

(

=

1 ,

2 )

∈

µ

_i

ν

_i

I

µ

,

if

µ

₁

( )

0 =

µ

₂

( )

0 =

0

, and at least one of

( ) ( )

0 ,

₂

0

1

ν

is not zero, and

d

,

s

are limited. And:

( )

_{( )}

( )

_{( )}

1 2

2 2

0 0

lim

0 , lim

0

α α

φ α

µν

α

→ +

≥

→ +

≥

.

Then

∃ ∈

α β

to satisfy the condition that:

( )

(

)

( )

,

(

0,

)

f x

α

≤

f x

+

µ

⎡

_⎣

F

α

−

f x

⎤

_⎦

α

∈

α

(6) Prove:

Let

µ

₁

( ) ( )

0 ,

µ

₂

0 ≠

0

, ∵

g

( )

α

∈

α

_β

f

(

x

( )

α

)

Based on the definition:

( )

x

f

(

x

( )

α

)

g

( )

α

(

x

( )

α

)

(4)

( )

(

x

) ( )

f

x

( ) ( )

g

s

( )

g

d

f

α

−

≤

φ

₁

α

T

α

+

φ

₂

0

T

[ ]

(

( )

) ( )

( ) ( )

( )

[

( ) ( )

]

[

F

f

x

g

s

g

d

F

f

x

]

x

f

x

f

Q

T

+

−

≤

−

=

α

µ

α

φ

α

φ

α

µ

α

2 1 Then

( )

(

) ( )

( )

1 0 0 2 0 1 2

lim

...

lim

1

0

T

T T

Q

g S

F

f x

g d

g s

g d

α α α

α

φ α

α

φ α

α

µ

α

µ µ

µ

→ + → + → +

≤

+

−

⎡

⎤

= −

_⎣

+

_⎦

<

If

µ

1

( )

0 =

µ

2

( )

0 =

0

, and at least one of

( ) ( )

0 ,

₂

0

1

ν

is not zero.

∵

g

( )

α

is continuous from the right of point

x

, that is

( )

α

→

g

(

α

→

0 +

)

g

, and

d

,

s

are limited

( )

T T

g

α

s

g s

∴

→

,

g

T

( )

α

d

→

g

T

d

(

α

→

0 +

)

∴If

ε

>

0

,

ζ

>

0

；

if

α ζ

<

, then

g

T

( )

α

s

≤

g

T

s

+

ε

,

( )

α

d

≤

g

d

+

ε

g

T T

( )

( ) ( )

( )

(

)

( )

1 2 1 2 T T

Q

g

d

g

s

F

f x

α

ϕ α

α

ϕ α

α

ϕ α

ϕ α ε µ

α

∴

=

+

−

⎡

_⎣

−

⎤

_⎦

And

g s

T

≤

0 ，

g d

T

≤

0

,

φ

₁

( )

α

≥

0 ，

φ

₂

( )

α

≥

0 (

α

∈

β

)

( )

1 2 2 0 0 2

1 ₀ 2 1

lim

0 lim

0

T T

Q

g S

g d

α α α

α

φ α

α

φ α

µν

α

→ + → + → +

∴

≤

⎛

⎞

−

+

−

⎜

⎟

⎜

_⎝

_⎠

⎟

⎝

⎠

<

Above all,

∃ ∈

α β

to let

α

∈

(

0,

α

]

and satisfy :

( )

(

x

) ( )

f

x

[

F

( ) ( )

f

x

]

f

α

−

≤

µ

α

−

.

Equation (6) is proved.

Based on Theorem 3, doing linear search along the curve

p

( )

α

can efficiently let the value of the function decrease, and in the meantime, its flexibility allow us to build various models according to the situations during calculating, so that it can reduce the occurrence of the phenomena so-called saw tooth .

IV. THE SELECTION OF

s

kAND

d

k

There are two methods of selecting

s

_kand

d

_k: 1) Let

s

_k

= −

g

_k,

s d

_kT _k

≥

0

(that is

g d

_kT _k

≤

0

) 2) Let

s

_k

= −

g

_k,

s

_kT

β

_k

d

_k

≥

0

(that is

g

_kT

β

_k

d

_k

≥

0

)

∵

2 2 2 2 2

( )

(

)

(

) (

)

(

)(

)

k

k k k k k

T T

k k k k k k k k k

k

k k k k

ared

f k

f x

s

d

g

s

d

s

d

x

s

d

α

=

−

+

=

+

∇

+

Where

x

k

=

x

k

+

θ α

(

_k2

s

_k

+

α

_k

d

_k

)

(

0 ≤ ≤

θ

1

)

∴

2 2 2 2

( ) ( ) ( )( )

1 1

2₍ ₎ 3 4

2 2 aredk r k _predk k T T

g_k _k s_k _{k k}d _k s_k _{k k}d f x _k s_k _{k k}d

T T T T T

g d g s d d s d s s

k k k k k k k k k k k k k k k k k

α α α α α α α α β α β α β = = + + + ∇ + + + + + Then:

i)where

g d

_kT _k

≺

0

, 0

lim

1

k k

r

α →

=

ii)where

g d

_kT _k

=

0,

g

_k

≠

0,

s

_k

= −

g

_k,

lim 0

1

2 ₍ 2 ₎ 2 ₍ ₎₍ 2 ₎ 2

lim

1 1

2 3 4

0 ₍ ₎

2 2 1 r k k k T

g s d f x s d

k k k k k k k k k k

T T _k T T

g s d d s d s s

k _k _k _k _k _k _k _k _{k k} _k _k _{k k}

α α α α α α α _α _β _α _β _α _β = → + + ∇ + → ₊ ₊ ₊ = iii)where

0

k k

s

= −

g

=

,

2 2 2

1 (

)

2

1

2

k T

k k k

k

T

k k k k

d

f x d

r

d

α

β

∇

=

0

lim

1

k k

r

α →

∴

=

So the method is trusted. Equations relative to

α

:

Let

w

=

α

2

s

_k

+

α

d

_k 2 3 4

1 ( )

(

)

2

1

2

T T T

k k k k k k k

T T

k k k k k k

g d

g s

d

s

d

s

α

β

α

β

α

β

Φ

=

+

Suppose

{

2

}

( )

α

min

( );

α α

s

_k

α

d

_k _k

Φ

=

Φ

+

≤ ∆

(P4)

Question (P4) equals to that:

∃ ≥

γ

0

, let:

2 2

[ ( )( )] (2 ) 0, (7.1)

, (7.2)

T

k k k k k k

k k k

g I d s s d

s d

β

γ

α

⎧ + + + + = ⎪ ⎨ ₊ _{= ∆} ⎪⎩

In question (P4), the constrained condition 2

k k k

s

d

α

+

α

≤ ∆

can be further strengthened to:

① Assume that

α

≥

0 ∵

2 2 2

k k k k k k

s

d

s

d

s

d

α

−

α

≤

α

+

α

≤

α

+

α

∴

2

k k k

s

d

(5)

Find the solutions of equation (8): 2

4

0

2

k k k k

k

d

s

α

−

+

+ ∆

≤ ≤

The

range of

α

is:

2

4

0

2

k k k k

k

d

s

α

−

+

+ ∆

≤ ≤

(9)

② Assume that

α

≤

0

2

k k k

s

d

α

−

α

≤ ∆

(8’) Find the solutions of equation (8’):

2

4

2

4

2

k k k k

k

k k k k

k

d

s

d

s

α

−

+ ∆

≤

+

+ ∆

≤

The range of

α

is:

2

4

0

2

k k k k

k

d

s

α

−

+ ∆

≤ ≤

(9’)

According to the above, the range of

α

is: 2

4

0

2

k k k k

k

d

s

α

−

+

+ ∆

≤

(7.1) can be transformed into

2 3

[2

]

3 [

] 2

[

]

0

T T T T

k k k k k k k k k

T T T T

k k k k k k k k k k

g d

g s

d d

d

s

d

s d

s

s s

α

γ

β

α

β

γ

α

β

γ

+

=

(10) Based on (10)

2

2[

] 3 3[

]

[2

]

0

T T T T

k k k k k k k k k k

T T T T

k k k k k k k k k

s

s s

s

d

s d

g s

d

d d

g d

β

γ

α

β

γ

α

β

γ

α

+

=

Let

,

2 ,

3[

],

2[

]

T

k k

T T T

k k k k k k k

T T

k k k k k

T T

k k k k k

a

g d

b

g s

d

d d

c

s

d

s d

d

s

s s

β

γ

β

γ

β

γ

=

+

=

+

=

+

Then According to formula giving roots of cubic equations:

3 3 2

2 3

3

2 2

3 3 2

2 3

3 3

2 2

1 2 1 2 1

[ ( )] [ ( )] [ ( )]

2 3 27 2 3 27 3 3

1 2 1 2 1

[ ( )] [ ( )] [ ( )]

2 3 27 2 3 27 3 3

( )

bc c bc c c

a a b

d d d d d d d d

bc c bc c c

a a b

d d d d d d d d

α

ϕ γ

= − + + − + + −

+ − − + + − + + −

=

s.t.

Φ

( )

γ

=

α

2

s

_k

+

α

d

_k

Based on Newton iteration method, we gain the root

γ

kof

Φ

( )

γ

− ∆ =

k

0

.

Then

α

_k

=

ϕ γ

2

(

_k

)

s

_k

+

ϕ γ

(

_k

)

d

_k

If we Select

s d

k

,

k s.t.

,

0

T

k k k k

s

= −

g g d

=

. And then (7) can be transformed into

2

[2

]

3[

]

2[

]

0

T T T

k k k k k k k

T T T T

k k k k k k k k k k

g s

d d

d

s

d

s d

s

s s

γ

β

α

β

γ

α

β

γ

+

=

⇒

When

α

≠

0

,

2

2 3[

]

2[

]

0

T T T T T

k k k k k k k k k k k k

T T

k k k k k

g s

d d

d

s

d

s d

s

s s

γ

β

γ

α

β

γ

α

+

=

According to the formula of extraction of root :

3

4[ ]

2

9( ) 8( )(2 )

4[ ]

3

4[ ]

2

9( ) 8( )( 2 )

4[ ]

3

T g_k _{k k}d

T T

s_k _{k k}s s_k s_k

T T T T T T

S_k _{k k}d s_k _{k k}s s_k s_k g_k s_k d_k d_k d_k _{k k}d

T T

sk k ks sk sk T

g_k _{k k}d

T T

g_k _{k k}g g_k g_k

T T T T T T

g_k _{k k}d g_k _{k k}g g_k g_k g_k g_k d_k d_k d_k _{k k}d

T T

gk k kg gk gk T

gk

β α

β γ

β β γ γ β

β γ β

β γ

β β γ γ β

β γ −

= ±

+

− + + +

+

= ±

+

− + − + +

+

= 9( )2 8 ( ) ( 2 ( )

4 [ ]

T T T T

d g d g I g g g d I d

k k k k k k k k k k k k k

T

g_k _k I g_k

β β β γ β γ

β γ

± − + − + +

+

Among this formula, (

±

) is determined by (9) and (9’) . And then (7.2) can be transformed into

α

2

d d

kT k

+

α

4

g g

kT k

= ∆

k

So we can select

s d

k

,

k_{conveniently to determine}

a

k_,

and to optimization searching calculate with algorithm 1or 2

ACKNOWLEDGMENT

This work was supported by Project 10971230 of the National Science Foundation.

REFERENCES

[1] Yuan Yaxiang, Sun Wenyu. Optimization theory and methods. Beijing:Science Press, 1997.

[2] J P Bulteau J P Vial, Curvilinear path and trust region in unconstrained optimization: a convergence analysis. Math. Prog. Study, 1987, 30:82-110

[3] Powell M J D. On the global convergence of trust region Algorithms for unconstrained minimization, Math. Prog, 1984, 29(3), 327-335

(6)

[5] More J J. Recent developments in algorithms and software for trust region method, in Mathematical Programming – the state of the art, Bachem A R, Grotschel M and Korte B eds. Springer Verlag, Berlin, 1983

[6] Sorensen D C. The trust region meths for unconstrained optimization. In Powell M J D ed. Nonlinear Optimization 1981, Academic Press, London, 1982, 29-38

[7] Sorensen D C. Newton’s method with a model trust region modification. SIAM J. Numer. Anal. 1982, 19(4):409-426 [8] Sorensen D C and More J J. On the use of directions of

negative curvature in a modified Newton method. Math. Prog. 1976, 16(1):1-20

[9] Shultz G A, Schnabel R B and Byrd R H. A family of trust region based algorithms for unconstrained minimization with strong gobal convergence properties. SIAM J. Numer. Anal., 1985, 22(1):47-67

[10] Yuan Y. On the convergence of trust region algorithms, Mathematica Numerica Sinica, 1996, 16:333-346

[11] Zhang Xiangsun, Chen hangwen, Zhang Juliang. A self-adaptive trust region method for unconstrained optimization, Or Transaction, 2001, 5(1):53-62

[12] Tang Jian, Duan Yurong. Trust region method for unconstrained optimization, J. of Chongqing University, 1991, 14(1): 33-41

[13] Fletcher R. Practical methods of optimization (Vol. 1, unconstrained optimization). Dundee:John Wiley & Sons Ltd., 1981

[14] Deng Naiyang. A negative curvature method and its convergence. Computing Mathematica, 1984, 6:23-28. [15] Powell M JD. A new algorithm for unconstrained

optimization, In Nolinear programming. Rosen J H, Mangasarian O L and Ritter K, eds. , New York： Academic Press , 1970

[16] Powell M J D. Convergence properties of a class of minimization algorithm”, in Nonlinear programming 2, Mangasarian O L, Meyer R R and Rolinson S M, eds. , New York：Academic Press, 1975

[17] Yang s p, Yuan X G.A trust region algorithm using curve-l inear searching direction for unconstrained optimization, Hunan University academic journal 2005.