Nondifferentiable Convex Optimization: An Algorithm Using Moreau-Yosida Regularization

Nondifferentiable Convex Optimization: An

Algorithm Using Moreau-Yosida Regularization

Nada DJURANOVIC-MILICIC 1, Milanka GARDASEVIC-FILIPOVIC2

Abstract—In this paper we present an algorithm for minimization of a nondifferentiable proper closed convex function. Using the second order Dini upper directional derivative of the Moreau-Yosida regularization of the objective function we make a quadratic approximation. The purpose of the paper is to establish that the sequence of points generated by the algorithm has an accumulation point which satisfies the first order necessary and sufficient conditions. A convergence proof is given, as well as an estimate of the rate of convergence.

Index Terms— Moreau-Yosida regularization, non-smooth convex optimization, second order Dini upper directional derivative.

I. INTRODUCTION

he following minimization problem is considered:

)

(

x

f

min

_n

R x∈

{ }

+

∞

∪

→

R

R

f

_:

n

*

, (1) where is a convex and not necessary differentiable function with a nonempty set

X

of minima.

For nonsmooth programs, many approaches have been presented so far and they are often restricted to the convex unconstrained case. It is reasonable because a constrained problem can be easily transformed to an unconstrained problem ussing a distance function. In general, the various approaches are based on combinations of the following methods: subgradient methods; bundle techniques and the Moreau-Yosida regularization.

For a function it is very important that its Moreau-Yosida regularization is a new function which has the same set of minima as and is differentiable with Lipschitz continuous gradient, even when is not differentiable. In [12, 13, 17] the second order properties of the Moreau-Yosida regularization of a given function are considered.

f

f

f

f

1

LC

Having in mind that the Moreau-Yosida regularization of a proper closed convex function is an function, we present an optimization algorithm (using the second order

Manuscript received February 24, 2012. This work was supported by Science Fund of Serbia, grant number III44007.

1 _{Nada DJURANOVIC-MILICIC is with Faculty of Technology and}

Metallurgy, University of Belgrade, Serbia (corresponding author to provide phone: +381-11-33-03-865; e-mail: [email protected]

2_{Milanka GARDASEVIC-FILIPOVIC is with Vocational College of}

Dini upper directional derivative (described in [1,2]) based on the results from [3]. That is the main idea of this paper.

We shall present an iterative algorithm for finding an optimal solution of problem (1) by generating the sequence of points

{ }

x

_k

0

,...,

1

,

0

2

1

=

+

+

=

≠

+ k k k k k k

k

x

s

d

k

d

x

α

α

k

of the following form:

(2) where the step-size

α

and the directional vectors and

are defined by the particular algorithms.

k

s

k

d

s

∇

Paper is organized as follows: in the second section some basic theoretical preliminaries are given; in the third section the Moreau-Yosida regularization and its properties are described; in the fourth section the definition of the second order Dini upper directional derivative and the basic properties are given; in the fifth section the semi-smooth functions and conditions for their minimization are described. Finally in the sixth section a model algorithm is suggested and its convergence is proved, and an estimate rate of its convergence is given, too.

II. THEORETICAL PRELIMINARIES

Throughout the paper we will use the following notation. A vector refers to a column vector, and denotes the gradient operator

Technology, Arandjelovac, Serbia; e-mail: [email protected]

T

n

x

x

x

⎟⎟

_⎠

⎞

⎜⎜

⎝

⎛

∂

∂

∂

∂

∂

∂

,...,

,

2 1

. The Euclidean product is denoted by

⋅

,

⋅

and

⋅

is the associated norm. For a given symmetric positive definite linear operator

M

we set

⋅

,

⋅

_M

:

=

M

⋅

,

⋅

; hence it is shortly denoted by

M

M

x

x

x

2

:

=

,

. The smallest and the largest eigenvalue of

M

we denote by

λ

and

Λ

respectively.

The domain of a given function

f

_:

R

n

→

R

∪

{ }

+

∞

_is

the set

dom

( )

f

=

{

x

∈

R

n

f

( )

x

<

+∞

}

_{. We say}

_f

_is

proper if its domain is nonempty. The point

( )

x

f

arg

x

*

=

A vector

g

∈

R

n to be a subgradient ven

min

n

R x∈

refers to the minimum point of a given function

f

_:

R

n

→

R

∪

{ }

+

∞

.

is said of a gi

proper convex function

f

:

R

n

→

R

∪

{ }

+

∞

at a point

( )

x

p

of t function (3), i. e.: he n

R

x

∈

if the inequality

f

( ) ( )

z

≥

f

x

+

g

T

⋅

(

z

−

x

)

holds for all

z

∈

R

n. The set of all subgradients of

f

( )

x

int

at the po

x

, called the subdifferential at the point

x

, is denoted by

∂

f

( )

x

. The subdifferential

∂

f

( )

x

is a nonempty se if and only if t

x

∈

dom

( )

f

. The cond tion

( )

x

f

∂

∈

0

is a first order necessary an fficient condition for a global minimiz vex function

f

n

R

x

∈

(see in [14,15]). For a convex function

f

it follows that

( )

(

)

i d su r

er fo the con at th

( )

e point

{

g

x

z

}

ma

x

f

₌

_x

_f

_z

₊

T

₋

_ho

R

z_∈ n lds, where

g

∈

∂

f

( )

z

eralization ions. (s

The of the subgradi for nondi ee in [10]).

s a simple gen of the gradi

concept ent

e fferenti

nt i

able convex funct

The directional derivative of a real function

f

defined on n

R

at the point

x

′

∈

R

n in the direction

s

∈

R

n, denoted

by

f

′

( )

x

′

,

s

, is

( )

(

) ( )

t

x

f

s

t

x

f

lim

s

x

f

0 t

′

−

⋅

function a

x

′

the direction

call th

+

′

=

′

′

↓

,

n this limit exi For a real convex dire rivative

s

exists in any direction

s

∈

R

(see in [2]).

At the end of this section we re definition of an 1

C

function.

whe sts.

at t

L

D

he poi

ctional de nt

n

n _in

e

R

∈

efinition 1. A real function

f

defined on

R

n is an

LC

1

on on the

functi open set

D

t

n _continuously

R

⊆

if it is

differentiable and its gradien

∇

f

is locally Lipschit ,

i.e.

( )

( )

zian

y

x

L

y

f

x

f

−

∇

≤

∇

r

x

,

y

∈

D

holds for

some

L

>

0

.

TH

−

-YOSIDA fo

III.

n 2

E M R

.

OREAU EGULARIZATION

Definitio Let

{ }

+

∞

be a proper closed

of a

f

etric defined by M,

∪

→

R

associated to the m

R

f

_:

n

, fined as

convex function. The Moreau-Yosida regularization

given functio denoted by F, is de

( )

n

⎭

M

utio

onv on defi

⎬

⎫

⎩

⎨

⎧

₊

₋

∈

2

1

)

(

:

R

y

f

y

y

x

min

x

F

=

_n . (3)

2

The above funct infimal convol

hat the in uti f a c

so a convex funct h i ned by (3) is

The min

ion is an fimal convol

ion. Hence t

n. In [10] it is ex function is proved t

al

a convex

on o e funct

function and has the same set of minima as the function

f

(see in [6]), so the motivation of the study of Moreau-Yosida regularization is due to the fact that

( )

x

f

min

n

R

x∈ is equal to

min

_x∈Rn

F

( )

x

.

imum point

( )

⎭

⎬

−

+

=

∈

2

)

(

:

_M

R y

x

y

y

argmi

x

p

n

⎫

⎩

⎨

⎧

1

2

f

n

(4)

is called the proximal point of

x

.

In [6] it is proved that the function

F

defined by (3) is

el

s,

F

has a Lipschitzian gradient on

th n

always differentiable.

The first order regularity of

F

is w l known: without any further assumption

e whole space

R

. More precisely,

( )

( )

( )

( )

2 1 2

holds for all

x

₁

,

x

₂

∈

R

n (see in [12]), where 1

2 2

1

F

x

F

x

F

x

,

x

x

x

F

−

∇

≤

Λ

∇

−

∇

−

∇

( )

x

M

x

p

x

f

(

p

( )

x

)

F

(

−

(

))

∈

∂

and

(

∇

=

p

x

)

is the

in (4).

consideration and Definition 1, we co that 1

Lemm

unique minimum So, according to above nclude

F

is an

LC

function.

a 1. [13]: The following statements are equivalent:

(i)

x

minimizes

f

; (ii)

p

( )

x

=

x

; (iii)

∇

F

( )

x

=

0

; (iv)

x

minimizes

F

; (v)

f

(

p

( )

x

) ( )

=

f

x

; (vi)

F

( )

x

=

f

(

x

)

.

. DINI SECOND U CTIONAL DERIVATIVE We shall give some preliminaries that will be used in the

D

IV PPER DIRE

remainder of the paper.

efinition 3. The second order Dini upper directional derivative of the function

f

∈

LC

1 at the point

x

∈

R

n in the direction

d

∈

R

nis defined to be

( )

[

(

)

( )

]

α

d

x

f

f

lim

d

x

f

T D

⋅

∇

−

∇

=

′′

,

α

α

d

x

sup

+

↓0

.

f

∇

is directionally differentiable at

x

_k, we have

(

(

)

If

(

)

(

)

[

)

]

α

x

f

x

f

lim

d

x

f

d

f

_D

′′

,

=

′′

_k

,

=

α

α

d

d

x

T k

⋅

∇

−

+

∇

↓0

for all

d

∈

R

n.

Since the Moreau-Yosida regularization of a proper convex unction

f

is an

LC

1 function, we can co

re

R

closed f

nsider itssecond order Dini upper directional derivative at the point

x

∈

R

n in the di ction

d

∈

n, i.e.:

( )

_x

,

_d

_limsup

g

1

g

2

_d

,

F

0

D

_α

α

−

=

′′

where

g

₁

∈

∂

f

(

p

(

x

+

α

d

)

)

,

g

₂

∈

∂

f

(

p

( )

x

)

, and

F

( )

x

is defined by (3) for

M

=

I

.

Lemma 2.[2]: Let conve er

function and

R

R

f

_:

n

_→

_{be a closed x prop}

F

is its Moreau –Yosida regularization for

I

M

=

ents are valid.

(

)

))

2

d

. Then the next statem

(i)

F

_D

(

x

_k

,

kd

)

k

F

_D

x

_k

,

d

)

(

(

1

)

(

2

1

2

,

,

d

d

F

x

d

F

x

x

+

≤

′′

+

′′

2

_′′

=

′′

(ii)

F

_D

′′

(

_{k D k D k}

,

(iii)

F

_D

′′

(

x

_k

,

d

)

≤

K

⋅

d

2, where

K

is some constant.

V. SEMI-SMOOTH FUNCTIONS AND OPTIMALITY CONDITIONS

Definition 4. A function

∇

F

:

R

→

R

is said t

the point

x

∈

R

n

n _{o be}

semi-smooth at if

∇

F

is locally d the limit r any

e that osed convex proper funct

gradient of its Moreau-Yosi s a

i-smooth functi

n

n

R

x

∈

an

)

exists fo Lipschitzian at

{ }

Vh

V

F

(

x

h

lim

d

h

λ

λ

+

∂

∈

↓ →

2

0

,

_d

_∈

_R

n_.

Not for a cl ion, the

da regularization i sem on.

Lemma 3. [16]: If

∇

F

:

R

n

→

R

n is semi-smooth at the

point

x

∈

R

n then

∇

F

is di and for any

rectionally differentiable at n

R

x

∈

V

∈

∂

2

F

(

x

+

h

)

,

h

→

0

we have:

) (

(

x

h

)

o

( )

h

Vh

−

′

,

=

. Similarly we have that

( )

F

∇

( )

2

h

Vh

h

−

.

Lemma 4.

,

h

o

x

F

′′

=

T

[4]

R

be a proper closed convex nction and l Moreau-Yosida regularization.

on em (1) then

( )

≥

0

′′

.

:Let

f

:

R

n

→

et

F

be its

n

R

is soluti fu

So, if

x

∈

of the probl

( )

,

=

0

′

x

d

F

and

F

_D

x

,

d

for all

d

∈

R

n.

Lemma 5 [4]

F

:L

oreau-Y

et

f

:

R

n

→

R

be a proper closed convex

its M a regula ,

function, osid rization and

x

a

point from

R

n. If

F

′

( )

x

,

d

=

0

and

F

′′

( )

x

,

d

>

0

for all n

R

).

VI. A

In this section an algorithm for so problem (1) is introduced. We suppose that at each it is possible

to com

D n

R

d

∈

, then

x

∈

is a strict local minimizer of the problem (1

MODEL ALGORITHM lving the

n

R

x

∈

pute

f

(

x

),

F

( )

x

,

∇

F

( )

x

and _D

′′

( )

x

,

d

for a gi

w er th wing b

( )

( )

( )

F

ven

d

∈

R

n.

At the k-th iteration e consid e follo pro lem

(

x

d

)

F

d

x

F

d

d

D k

T k k

k

n

2

,

1

_′′

+

∇

=

Φ

Φ

, (5)

where

min

R

d∈

,

(

x d

)

FD′′ k, stands for the second order Dini upper

directional derivative at

x

_k in the direction

d

. Note that if is a Lipschitzian constant for

F

, it is also a Lipschitzian

for

Λ

constant

∇

F

. The function

Φ

( )

d

is called an iteration function. It is easy to see that

k

( )

0

=

0

Φ

_k and

( )

d

k

Φ

is Lipschitzian on

R

n. We generate the sequence

{ }

x

k of the form

0

,

0

,

2

1

=

+

+

≠

≠

+ k k k k k k

k

x

s

d

s

d

x

α

α

_k ,

where the step-size

α

_k and the direction vectors

s

_k and k

d

are defined by particular algorithms.

(

For a given

q

∈

0

,

1

)

the step-size

α

_kis a number sfying

sati ( ) e smallest integer

from

k i k

=

q

α

, where

i

( )

k

is th

{

0

,

1

,

2

,...

}

such that

( ) ( )

( )

_{( )}

( )

₍

₍

₎

₎

_⎟

⎠

⎞

⎜

⎝

⎛

₋

_′′

≤

≤

∇

−

+

T k k

s

x

F

x

F

₁

k k D k i k

k i

k

d

x

F

q

q

x

F

,

2

1

4

σ

(6)

and where

σ

∈

( )

0

,

1

is a preassigned constant, and n

R

x

₀

∈

is a given point.

ollowing A1. Sup

We make the f assumptions.

pose that ₁

≤

F

_D

′′

(

x

_k

,

d

)

≤

c

₂

d

2 hold for for every

d

∈

R

n. 2

d

c

some

c

₁ and

c

₂ such that

0

<

c

₁

<

c

₂ A2.

d

_k

=

1

,

=

1

,

k

=

0

,

1

,

2

,...

a value

s

_k

A3. There exists

β

>

0

such th at

( )

x

T

s

_k

≤

−

∇

F

( )

x

_k

⋅

s

_k

,

k

0

,

1

,

2

,...

k

β

=

∇

F

Lemma 6. [4]: Under the assumption A1 the function

( )

Φ ⋅

_k is coercive.

ptima

also m nder the assumption A1 the direction se

Remark. Coercivity of the function

Φ

_k assures that the

o l solution of the problem (5) exists (see in [16]). It eans that, u

Proposition 1. [3]: If the Moreau-Yosida regularization

( )

⋅

F

of the proper closed convex function

f

( )

⋅

satisfies assumption A1 then :

conv

( )

(i) the function

F

( )

⋅

is uniformly and, hence, strictly ex;

(ii) the level set

( )

₀

{

( )

₀

}

compact convex se d

(iii) there exi

:

F

x

F

R

x

x

₌

_∈

n

_≤

_x

_{is a}

t, an

sts a unique point

x

* such that

L

( )

x

min

_{( )}

F

( )

F

x L x 0 * ∈

=

x

.

Lemma 7. [4]: The following statements are equivalent:

ly

(i)

d

=

0

is the global optimal solution of the problem (5)

i)

0

is the optimum of the objective function in (5)

(i

(iii) the corresponding

x

_k is such that

0

∈

∂

f

( )

x

_k

Convergence theorem. Suppose that

f

is a proper closed

ts M da regularization

F

tisfies mptions A1, A2 A3. Then for any initial convex funct

sa

ion and i oreau-Yosi

assu and

point

x

∈

R

n

_,

x

_k

→

x

_∞

0 , as

k

→

+∞

, where

x

∞ is a unique minimal point of the function

f

.

Proof. If

d

_k

≠

0

is a n of (5 ws

tl we have by

assumption A1 that

solutio ), it follo that

( )

_k

0

_k

( )

0

k

d

≤

=

Φ

Φ

. Consequen y,

( )

(

,

)

1 0

2

2 1 <

− ≤

′′ _{k k k}

D k

k x d ₂c d . (7)

1 − ≤ ∇ T F d x F

From the above inequality it follows that the vector a descent direction at y (6) and assumption A1 we get

k

d

is k

x

. B

( ) ( )

( )

_{( )}

( )

₍

₎

( )

( ) ₀

2

1 2

1 4 _⎟<

⎠ ⎞ ⎜ ⎝ ⎛_{− ∇ ⋅ −} ≤ ∇ k k i k

k s q c d

x F F β σ

0

≠

. Hence the sequence

{

F

( )

x

_k

}

descent y, and, consequent he sequence

)

L

( )

x

₀ is by the Proposition 1 a

uence , 2 1 4 1 ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ _{− ′′} ≤ −

+ D k k

k i k T k k i k

k Fx q x s q F x d

x

F σ

(8)

for every

d

_k propert

}

⊂

L

(

x

₀

has the ly, t

eq

{

x

_k . Since

compact convex set, it follows that the s

{ }

x

_k is bounded. Therefore there exis ccumulation points of the

.

Since

∇

F

is continuous, then, if

( )

→

→

+∞

∇

F

x

_k

,

k

t a

}

then it follows that every

x

of th

{ }

x

satisfies

a poi ₀ such that

sequence

{

x

_k

0

lation point

0

=

∞ . Since here exists accumu

∇

F

x

convex, t

∞ k

( )

F

is (by the Proposition 1) strictly

( )

x

L

x

∈

e sequence

nt _∞ unique

( )

=

0

∇

F

x

_∞ . Hence, the sequence

{ }

x

_k a unique ∞ and it i a global minimizer of

F

and by Lemma 1 it is a global minimizer of th

f

.

e have to prove that

( )

has limit point

x

s

e function

Therefore w

∇

F

x

0

,

k

→

+∞

.

Let

K

₁ be set of indices such that _∞ ∈

→

k

a

=

x

a) The set of indices

x

lim

_k

K k 1

. Then there are two cases to consider:

{ ( )}

i

k

for

k

∈

K

₁, is uniformly bounded above by a number

I

. From (6) it follo at:

A2, A3 and ws th

( ) ( )

)

( )

₍

( )

( )

(

)

(

)

( )

(

)

( )

(

,

)

.

2

1

,

2

,

2

1

4 4 4 1 k k D I k I k k D I k k I k k D I k T k I k i T k k

d

x

F

q

x

F

q

d

x

F

q

s

x

F

q

d

x

F

q

s

x

F

q

x

F

x

F

′′

−

∇

−

≤

′′

−

⋅

∇

≤

⎟

⎠

⎞

⎜

⎝

∇

−

′′

′′

,

2

1

4 k k D k k k

i

_F

_x

_s

_q

_F

_x

_d

q

⎛

≤

⎟

⎠

⎞

⎜

⎝

⎛

_∇

₋

≤

−

≤

+

σ

βσ

σ

β

σ

Hence, it follows that

σ

σ

( ) ( )

( )

(

,

)

.

2

1

4 1 k k D I I k k

d

x

F

q

x

F

q

x

F

x

F

′′

+

∇

≥

≥

−

₊

σ

βσ

(9)

Since

{ ( )}

F

x

_k is bounded bellow and

( ) ( )

x

_k+1

−

F

x

_k

→

0

F

as

k

→

∞

,

k

∈

K

₁, fro

follows that

m (9) it

( )

→

0

∇

F

x

_k and

F

_D

′′

(

x

_k

;

d

_k

)

→

0

,

k

→

∞

k

∈

K

₁ i.e.

x

_∞ is a stationary point of the objective function

( )

0

F

, i.e.

∇

F

x

_∞

=

unique optimal point of t ion

f

.

b) Th a

. From Lemma 1 it follows that

x

_∞ is a he funct

ere is subset

K

₂

⊂

K

₁ such that

( )

=

+∞

∞

→

i

k

lim

k .

By the definition of

i

( )

k

, we have for

k

∈

K

₂ that

( ) ( )

( )

_{( )}

( )

₍

₎

⎠

⎝

⎛

_⎟

⎞

⎜

∇

−

′′

>

>

−

− − + k k i k k i k k

2

4 4 1 1 . k D T

k

s

q

F

x

d

x

F

q

x

F

x

,

1

σ

. (10)

By Definitin 3, A1 and Lemma 2 we have

F

( ) ( )

( )

_{( )}

( )

_{( )}

( ) ( )

(

_,

1 2 2

)

2

1

_′′

−

₊

−

₊

+

(

2( ) 2

)

2 2 1 1 − − − +

+

∇

+

∇

=

−

=

k i k k T k i k T k k i k k

q

d

x

F

q

s

x

F

q

x

F

x

F

k k i k k i k

D

x

q

s

q

d

o

( ) ( ) ( )

( )

( )

_{( )}

( )

(

)

(

( )

)

(

( )

)

( )

( )

_{( )}

( )

₍

₎

( )

₍

₎

(

( )

)

( )

( )

_{( )}

( ) ( )

(

( )

)

_.

,

,

,

,

2 2 2 4 4 2 2 2 2 2 2 1 2 2 4 4 2 2 2 1 2 2 2 2 1 2 2 1 − − − − − − − − − − − −

+

+

+

∇

+

∇

+

′′

+

′′

+

∇

+

∇

+

′′

+

∇

+

+

∇

k i k k i k k

i k k

T k i k T k k i k k D k i k k D

k k k

T k i k T k k i k k i k D k k i k k k T k i k T k

q

o

d

q

c

s

d

x

F

q

s

x

F

q

o

d

x

F

q

s

x

F

x

s

q

F

x

d

F

q

o

d

q

x

F

s

q

x

d

x

F

q

s

x

F

2 2 − − −

+

≤

+

=

′′

+

≤

k i i k i D k i

q

c

q

q

q

F

q

Hence, from (10) it follows that:

( )

_{( )}

( )

₍

₎

( )

_{( )}

( )

_{( )}

( ) ( )

(

( )

)

_.

,

2

1

2 2 2 4 2 2 2 2 1 4 4 1 − − − − − −

+

∇

⎟

⎠

⎞

⎜

⎝

⎛

_∇

₋

_′′

k i k k k k T k i T k i k k D k i k T k k i

q

o

d

d

x

F

d

x

F

q

s

x

F

q

σ

Accumulati order higer than 4

2 2

2 −

₊

+

+

∇

≤

k i k

i

_s

k k

_c

_q

q

c

q

s

x

F

q

ng all terms of

o

(

q

2i( )k−2

)

to the ( )

in

(

2

)

_{(by assum}

0

≤

from th 2ik−

q

o

pti ng the fact that

( )

∇

_F

T

_x

_k

_d

_{llows that:}

on A2), and usi equality it fo

k e last in

( )

_{( )}

( )

_{( )}

( ) 2 2

(

2( ) 2

)

2 2 −

₊

+

∇

<

∇

k k

i k k

k k

s

q

c

s

x

F

q

s

x

F

q

σ

, 1 1 − − −

₊

k i T k i T k i

q

o

i.e.

(

)

_q

i( )k−1

_∇

_F

T

_{( )}

_x

_s

₊

_c

_q

2i( )k−2

_s

2

₊

_o

(

_q

2i( )k−2

)

_>

₀

.

1

−

σ

_{k k 2 k}

Hence , dividing by

c

₂

⋅

q

i( )k−1, by A2 and A3 it follows that

( )

_{( )}

( )

( )

( )

( )

( )

.

1

_o

_q

ik 1

k −

+

−

≥

β

1

2 2 2 1 2 1

c

x

F

c

c

q

o

s

x

F

c

q

ik− T _{k k} ik−

∇

+

∇

−

>

σ

σ

Since

q

i( )k−1

→

0

as

k

→

∞

,

k

∈

K

₂, it follows that

( )

→

0

as

k

→

∞

,

∇

F

x

_k

In order to have a fi value 2

K

∈

.

k

nite

i

( )

k

fficient th and

d

_k have descent

( )

x

_k T

s

_k

<

∇

_k

, it is su

∇

F

at

s

_k propert

whenever

ies, i.e.

0

and

( )

x

F

T _k

<

0

d

∇

F

( )

x

_k

≠

0

. ion

The first relat follows from A3 and the second rel f

t a omes

( )

ation ollows from (7).

A saddle point the relation (6) bec

( )

_x

_≤

₋

_q

4i(k)

_F

(

_x

_d

)

F

x

F

₋

−4

_′′

,

k k D k k

2

1 +

σ

In the case

d

_k

≠

0

by Lemma 7 and hence, by assumption A1 it follows that

F

_D

′′

(

x

_k

,

d

_k

)

>

0

; so (11) can be clearly satisfied.

Convergence rate theorem.

. (11)

Under the assum we have t

nce

ptions of the previous theorem hat the following estimate holds for the seque

{ }

_k generated by the algorithm.

( )

x

( )

( ) ( )

( )

1 1 2 1 2 0 0

1

1

− − + ∞

⎥

⎤

⎢

⎡

₋

+

≤

−

_∑

n k k

n

x

F

x

F

x

F

x

μ

μ

.

0

=

⎥⎦

⎢⎣

η

k

∇

F

x

_k

for

1

,

2

,

3

,..

F

=

n

, where

μ

₀

=

F

( )

x

₀

−

F

( )

x

_∞ and

( )

x

₀

= <

η

+∞

L

diam

(since by Proposition 1 it follows that

( )

x

0

L

is bounded).

Proof . The proof directly follows from the Theorem 9.2, page 167, in [11].

The Moreau-Yosida regularization is a powerful tool for

optimization problem using the properties of this regularization.

To our knowl h to solving NDO

p

[3] Djuranovic Milicic Nada: “ An Algorithm For _LC1 _{Optimization”,}

Yugoslav Journal of Operations Research, Vol. 15, No. 2, pp 301-306, 2005

[4] Djuranovic Milicic N., Gardasevic Filipovic M..: „An Algorithm For

r Minimization of a Nondifferentiable Convex Function“, Filomat, in press

VII. CONCLUSION

smoothing nondifferentiable functions. It allows us to transform the solving an NDO problem into the solving an

1

LC

edge this is a new approac

roblems, and in some sense it is close to the proximal quasi Newton algorithm.

The algorithm presented in this paper is based on the algorithm from [3]. This method uses minimization along a plane defined by the vectors

s

_k and

d

_k to generate a new iterative point at each iteration. Relating to the algorithm in [3], the presented algorithm is defined and converges for noonsmooth convex function.

REFERENCES

[1] Browien J., Lewis A.: “Convex analysis and nonlinear optimization”, Canada 1999

[2] Clarke F. H.: “Optimization and Nonsmooth Analysis”, Wiley, New York, 1983

Minimization Of A Nondifferentiable Convex Function“, Lecture Notes in Engineering and Computer Science, WCE 2009, VOL. II, pp 1241-1246, 2009

[5] Djuranovic Milicic N., Gardasevic Filipovic M. :“A Multi-step Curve Search Algorithm in Nonlinear Optimization - Nondifferentiable Convex Case" , FACTA UNIVERSITATIS Series Mathematics and Informatics, Vol. 25, pp. 11-24, (2010)

[6] Fuduli A.: “Metodi Numerici per la Minimizzazione di Funzioni Convesse Nondifferenziabili”, PhD Thesis, Departimento di Electronica Informatica e Sistemistica, Univerzita della Calabria (1998)

Fletcher R.: “Pr

[7] actical Methods of Optimization, Volume 2, Constrained Optimization”, Department of Mathematics, University of Dundee, Scotland, U.K,Wiley-Interscience, Publication, (1981) [8] Gardasevic Filipovic M., Djuranovic Milicic N. „An Algorithm Using

1239-1251, 2011

w,

Journal of

Numerical Functional Analysis and Optimization , VOL 25; No.5/6, pp 515-530, 2004

4] ПшеничнйБ. Н.: „Выпуклыйанализиэкстремальные задачи“,

Москва „Наука“, 1980.

5] Rockafellar R.: “Convex Analysis”, Princeton University Press, New Jersey (1970) (Russ. trans.: Mir, Moscow 1973)

6] Sun W., Sampaio R.J.B., Yuan J.: “Two Algorithms for _LC1

Unconstrained Optimization”, Journal of Computational athematics, Vol. 18, No. 6, pp. 621-632, 2000

[17] [9] Gardasevic Filipovic M. , Djuranovic Milicic N. "An Algorithm Using

Trust Region Strategy For Minimization Of A Nondifferentiable Function", Numerical Functional Analysis and Optimization, Vol. 32,

No. 12, pp. [1

[10] ИоффеА. Д., ТихомировВ. М.: „Теорияэкстремальныџзадач“,

Москва „Наука“, 1974

[11] Karmanov V.G: “Mathematical Programming”. Nauka, Mosco [1

[1 1975 (in Russian).

[12] Lemarechal C. , Sagastizabal C.: “Practical aspects of the

Moreau-Yosida regularization I: theoretical preliminaries”, SIAM M

Qi L.: “Second Order Analysis of the Moreau-Yosida

of Mathematics, The University of New Optimization, Vol.7, pp 367-385, 1997

[13] Meng F.,Zhao G.: ”On Secon -order Properties of the Morea -Yosida Regularization for Constrained Nonsmooth Convex Programs” ,

Regularization”, School