A Penalization Gradient Algorithm for Variational Inequalities

(1)

Volume 2011, Article ID 305856,12pages doi:10.1155/2011/305856

Research Article

A Penalization-Gradient Algorithm for

Variational Inequalities

Abdellatif Moudafi

1

_{and Eman Al-Shemas}

2

1_{D´epartement Scientifique Interfacultaires, Universit´e des Antilles et de la Guyane, CEREGMIA,}

97275 Schoelcher, Martinique, France

2_{Department of Mathematics, College of Basic Education, PAAET Main Campus-Shamiya, Kuwait}

Correspondence should be addressed to Abdellatif Moudafi,

[email protected]

Received 11 February 2011; Accepted 5 April 2011

Academic Editor: Giuseppe Marino

Copyrightq2011 A. Moudafi and E. Al-Shemas. This is an open access article distributed under

the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

This paper is concerned with the study of a penalization-gradient algorithm for solving variational

inequalities, namely, findx ∈ Csuch thatAx, y−x ≥ 0 for ally ∈ C, whereA : H → H

is a single-valued operator,Cis a closed convex set of a real Hilbert spaceH. GivenΨ : H →

Ê ∪ {∞}which acts as a penalization function with respect to the constraintx ∈ C, and a

penalization parameterβk, we consider an algorithm which alternates a proximal step with respect

to∂Ψand a gradient step with respect toAand reads asxk Iλkβk∂Ψ−1xk−1−λkAxk−1.

Under mild hypotheses, we obtain weak convergence for an inverse strongly monotone operator and strong convergence for a Lipschitz continuous and strongly monotone operator. Applications to hierarchical minimization and fixed-point problems are also given and the multivalued case is reached by replacing the multivalued operator by its Yosida approximate which is always Lipschitz continuous.

1. Introduction

LetHbe a real Hilbert space,A:H → Ha monotone operator, and letCbe a closed convex

set inH, we are interested in the study of a gradient-penalization algorithm for solving the

problem of findingx∈Csuch that

Ax, y−x≥0 ∀y∈C, 1.1

or equivalently

(2)

whereNC is the normal cone to a closed convex setC. The above problem is a variational

inequality, initiated by Stampacchia1, and this field is now a well-known branch of pure

and applied mathematics, and many important problems can be cast in this framework.

In2, Attouch et al., based on seminal work by Passty3, solve this problem with a

multivalued operator by using splitting proximal methods. A drawback is the fact that the

convergence in general is only ergodic. Motivated by2,4and by5where penalty methods

for variational inequalities with single-valued monotone maps are given, we will prove that

our proposed forward-backward penalization-gradient method1.9enjoys good asymptotic

convergence properties. We will provide some applications to hierarchical fixed-point and optimization problems and also propose an idea to reach monotone variational inclusions.

To begin with, see, for instance6, let us recall that an operator with domainDT

and rangeRTis said to be monotone if

u−v, x−y≥0 wheneveru∈Tx, v∈Ty. 1.3

It is said to be maximal monotone if, in addition, its graph, gphT : {x, y ∈H×H :y ∈

Tx}, is not properly contained in the graph of any other monotone operator. An operator

sequence Tk is said to be graph convergent toT if gphTk converges to gphT in the

Kuratowski-Painlev´e’s sense, that is, lim sup_kgphTk⊂gphT⊂lim infkgphTk. It is

well-known that for eachx ∈ H and λ > 0 there is a unique z ∈ H such thatx ∈ I λTz.

The single-valued operatorJ_λT: IλT−1is called the resolvent ofTof parameterλ. It is a

nonexpansive mapping which is everywhere defined and is related to its Yosida approximate,

namelyTλx : x−J_λTx/λ, by the relationTλx ∈TJ_λTx. The latter is 1/λ-Lipschitz

continuous and satisfiesTλμ Tλμ. Recall that the inverseT−1ofT is the operator defined

byx∈T−1y⇔y∈Txand that, for allx, y∈H, we have the following key inequality

JT

λx−JλT

y2≤x−y2_I₋_J_T

λ

x−I−JT

λ

y2. 1.4

Observe that the relationTλ_μx Tλμxleads to

JμTλx _λλ_μx 1−_λλ_μ

JT

λμx. 1.5

Now, given a proper lower semicontinuous convex function f : H → Ê ∪ {∞}, the

subdiﬀerential offatxis the set

∂fx u∈H:fy≥fx u, y−x∀y∈H. 1.6

Its Moreau-Yosida approximate and proximal mappingfλand proxλfare given, respectively,

by

fλx inf

y∈H

fy _2λ1 y−x2_,

proxλfx argmin

y∈H

(3)

We have the following interesting relation∂f_λ ∇fλ. Finally, given a nonempty closed

convex setC⊂H, its indicator function is defined asδCx 0 ifx∈Cand∞otherwise.

The projection ontoCat a pointuisPCu infc∈Cu−c. The normal cone toCatxis

NCx {u∈H:u, c−x ≤0∀c∈C} 1.8

ifx∈Cand∅otherwise. Observe that∂δC NC, proxλf J_λ∂f, andJ_λNC PC.

Given somexk−1 ∈ H, the current approximation to a solution of1.2, we study the

penalization-gradient iteration which will generate, for parametersλk > 0, βk → ∞,xkas

the solution of the regularized subproblem

1

λkxk−xk−1 Axk−1βk∂Ψxk0, 1.9

which can be rewritten as

xkIλkβk∂Ψ−1xk−1−λkAxk−1. 1.10

Having in view a large range of applications, we shall not assume any particular structure or

regularity on the penalization functionΨ. Instead, we just suppose thatΨis convex, lower

semicontinuous andCargminΨ_/∅. We will denote by VIA, Cthe solution set of1.2.

The following lemmas will be needed in our analysis, see for example 6, 7,

respectively.

Lemma 1.1. LetT be a maximal monotone operator, thenβkTgraph converges toN_T−1₀asβk →

∞provided thatT−1₀_/_∅.

Lemma 1.2. Assume thatαkandδkare two sequences of nonnegative real numbers such that

αk1≤αkδk. 1.11

If limk→∞δk 0, then there exists a subsequence of αk which converges. Furthermore, if

_∞

k0δk<∞, then limk→∞αkexists.

2. Main Results

2.1. Weak Convergence

Theorem 2.1. Assume that VIA, C/∅,Ais inverse strongly monotone, namely

Ax−Ay, x−y≥ _L1Ax−Ay2 _{∀x, y}_∈_H,

(4)

If

∞

k0

x−J_λkβk∂Ψx−λkAx<∞ ∀x∈VIA, C, 2.2

andλk ∈ε,2/L−ε(whereε >0 is a small enough constant), then the sequencexk_k∈generated

by algorithm1.9converges weakly to a solution of Problem1.2.

Proof. Letxbe a solution of1.2, observe thatxsolves1.2if and only ifx IλkNC−1x−

λkAx PCx−λkAx. Setxk Iλkβk∂Ψ−1_x₋_{λkAx, by the triangular inequality, we}

can write

xk−x ≤ xk−xkxk−x. 2.3

On the other hand, by virtue of1.4and2.1, we successively have

xk−xk2_{≤ xk−}

1−x−λkAxk−1−Ax2− xk−1−xk−λkAxk−1−Ax xk−x2

≤ xk−1−x2−λk 2_L−λk

Axk−1−Ax2

− xk−1−xk−λkAxk−1−Ax xk−x2.

2.4

Hence

xk−x_<_xk−₁₋_x2₋_ε₂_Axk−

1−Ax2− xk−1−xk−λkAxk−1−Ax xk−x2

x−xk.

2.5

The later implies, by Lemma1.2and the fact that2.2insures lim_k_→_∞x−x_k 0,

that the positive real sequencexk−x2

k∈converges to some limitlx, that is,

lx lim

k→∞xk−x

2_<_∞, _2.6

and also assures that

lim

k→∞Axk−1−Ax

2_0,

lim

k→∞xk−1−xk−λkAxk−1−Ax xk−x

2_0. 2.7

Combining the two latter equalities, we infer that

lim

k→∞xk−1−xk

(5)

Now,1.9can be written equivalently as

xk−1−xk

λk Axk−Axk−1∈

Aβk∂Ψxk. 2.9

By virtue of Lemma1.1, we haveβk∂Ψgraph converges toNargminΨbecause

∂Ψ−1₀_∂Ψ∗₀_argminΨ. _2.10

Furthermore, the Lipschitz continuity ofAsee, e.g.,8clearly ensures that the sequence

Aβk∂Ψgraph converges in turn toANargminΨ.

Now, letx∗be a cluster point of{xk}. Passing to the limit in2.9, on a subsequence

still denoted by{xk}, and taking into account the fact that the graph of a maximal monotone

operator is weakly strongly closed inH×H, we then conclude that

0∈AN_Cx∗, 2.11

becauseAis Lipschitz continuous,xkis asymptotically regular thanks to2.8, andλkis

bounded away from zero.

It remains to prove that there is no more than one cluster point, our argument is classical and is presented here for completeness.

Letxbe another cluster of{xk}, we will show thatx x∗. This is a consequence of

2.6. Indeed,

lx∗ _lim

k→∞xk−x

∗2_, _l_x _lim

k→∞xk−x

2_, _2.12

from

xk−x 2xk−x∗2x∗−x 22xk−x∗, x∗−x, 2.13

we see that the limit ofxk −x∗, x∗−x ask → ∞must exists. This limit has to be zero

becausex∗is a cluster point of{xk}. Hence at the limit, we obtain

lx lx∗ _x∗₋_x 2_. _2.14

Reversing the role ofxandx∗, we also have

lx∗ _l_x _x∗₋_x 2_. _2.15

That isxx∗, which completes the proof.

Remark 2.2. iNote that, we can remove condition2.2, but in this case we obtain that there

exists a subsequence of xk such that every weak cluster point is a solution of problem

(6)

βk∂Ψgraph converges to∂δC. The later is equivalent, see for example6, to the pointwise

convergence ofJ_λkβk∂ΨtoJ_λ∂δC∗ and therefore ensures that

lim

k→∞

x−J_λkβk∂Ψx−λkAx0. 2.16

iiIn the special caseΨx 1/2distx, C2,2.2reduces to∞_k₀1/βk < ∞, see

Application2of Section3.

Suppose now that Ψx distx, C, it well-known that proxγΨx PCx if

distx, C≤γ. Consequently,

J_λkβk∂Ψx PCx if distx, C≤λkβk, 2.17

which is the case for allk≥κfor someκ∈Æ becauseλkis bounded and limk→∞βk ∞.

Hence limk→∞x−J_λkβk∂Ψx−λkAx0, for allk≥κ, and thus2.2is clearly satisfied.

The particular caseΨ 0 corresponds to the unconstrained case, namely,C H. In

this context the resolvent associated toβk∂Ψis the identity, and condition 2.2is trivially

satisfied.

2.2. Strong Convergence

Now, we would like to stress that we can guarantee strong convergence by reinforcing

assumptions onA.

Proposition 2.3. Assume thatAis strong monotone with constantα >0, that is,

Ax−Ay, x−y≥αx−y2 _{∀x, y}_∈_H,

for someα >0, 2.18

and Lipschitz continuous with constantL >0, that is,

_Ax₋_Ay_≤_L_x₋_y _{∀x, y}_∈_H, for someL >0. 2.19

Ifλk∈ε,2α/L2₋_ε_(where_{ε >}_{0 is a small enough constant) and lim}

k→∞λkλ∗>0, then the

sequence generated by1.9strongly converges to the unique solution of 1.2.

Proof. Indeed, by replacing inverse strong monotonicity ofA by strong monotonicity and

Lipschitz continuity, it is easy to see from the first part of the proof of Theorem2.1that the

operator ofI−λkAsatisfies

_I₋_λ_k_Ax₋_I₋_λ_k_A

y2 _≤

1−2λkαλ2kL2

(7)

Following the arguments in the proof of Theorem2.1to obtain

xk−x ≤1−2λkαλ2

kL2xk−1−xδkx withδkx:x−J_λkβk∂Ψx−λkAx.

2.21

Now, by setting Θλ √1−2λαλ2_L2_{, we can check that 0} _< _Θλ _< _{1 if and only}

if λk ∈0,2α/L2_{, and a simple computation shows that 0} _< _Θλk _≤ _Θ∗ _< _{1 with}

Θ∗_max{Θε,_Θ2α/L2₋_{ε}. Hence,}

xk−x ≤Θ∗k_x

0−x

k−1

j0

Θ∗j_δk−jx. _2.22

The result follows from Ortega and Rheinboldt9, page 338and the fact that limk→∞δkx

0. The later follows thanks to the equivalence between graph convergence of the sequence of

operatorsβk∂Ψto∂δCand the pointwise convergence of their resolvent operators combined

with the fact that limk→∞λkλ∗.

3. Applications

(1) Hierarchical Convex Minimization Problems

Having in mind the connection between monotone operators and convex functions, we may

consider the special caseA ∇Φ,Φ being a proper lower semicontinuous diﬀerentiable

convex function. Diﬀerentiability ofΦensures that∇Φ NargminΨ∂Φ δargminΨand1.2

reads as

min

x∈argminΨΦx. 3.1

Using definition of the Moreau-Yosida approximate, algorithm1.9reads as

xkargmin

y∈H

fy_2λk1 y−I−λkAxk−12

. 3.2

In this case, it is well-known that the assumption 2.1 of inverse strong monotonicity of

∇Φis equivalent to its L-Lipschitz continuity. If further we assume ∞_k₁δkx < ∞ for

(8)

of algorithm 3.2 to a solution of 3.1. The strong convergence is obtained, thanks to

Proposition2.3, if in additionΨis strongly convexi.e., there isα >0;

1−μΨx1 μΨx2≥Ψ

1−μx1μx2

α

2μ

1−μx1−x22 3.3

for allμ∈0,1, allx1, x2∈Handλka convergent sequence withλk∈ε,2α/L2−ε. Note

that strong convexity ofΨis equivalent toα-strong monotonicity of its gradient. A concrete

example in signal recovery is the Projected Land weber problem, namely,

min

x∈CΦx:

1

2Lx−z

2_, _3.4

Lbeing a linear-bounded operator. SetAx:∇Φx L∗Lx−z. Consequently,

∀x, y∈H Ax−AyL∗_L_x₋_y_{≤ L}2_x₋_y_, _3.5

andAis therefore Lipschitz continuous with constantL2_{. Now, it is well-known that the}

problem possesses exactly one solution ifLis bounded below, that is,

∃κ >0 ∀x∈H Lx ≥κx. 3.6

In this case, A is strongly monotone. Indeed, it is easily seen that f is strongly convex:

considerx, y∈Handμ∈0,1, one has

_μLx₋_z

1−μLy−z2

2 ≤

μLx−z2

2

1−μLy−z2

2 −

κ2_μ₁₋_μ_x₋_y2

2 .

3.7

(2) Classical Penalization

In the special case whereΨx 1/2distx, C2, we have

∂Ψx x−Proj_Cx, 3.8

which is nothing but the classical penalization operator, see10. In this context, taking into

account the fact that

(9)

and thatxsolves1.2, and thusxPCx−λkAx, we successively have

xk−xJ_λkβk∂Ψx−λkAx−J_λkNCx−λkAx

λkβk∂Ψ_λkx−λkAx−NC_λkx−λkAx

λkβk∂Ψλkβkx−λkAx− ∇δCλkx−λkAx

λkβk∂δC1λkβkx−λkAx− ∇δCλkx−λkAx

λkβk∇δC1λkβkx−λkAx− ∇δCλkx−λkAx

λk _λk1 − βk

1λkβk

x−λkAx−PCx−λkAx

1

1λkβkλkAx ≤

1

βkAx.

3.10

So condition on the parameters reduces to∞_k₁1/βk < ∞, and algorithm1.9is nothing

but a relaxed projection-gradient method. Indeed, using1.5and the fact that J_λNC PC, we

obtain

xk ₁1_λ

kβkI

λkβk

1λkβkPC

I−λkAxk−1. 3.11

An inspection of the proof of Theorem2.1shows that the weak converges is assured with

λk∈ε,2/L−ε.

(3) A Hierarchical Fixed-Point Problem

Having in mind the connection between inverse strongly monotone operators and nonexpansive mappings, we may consider the following fixed-point problem:

I−PxNCx0, 3.12

withPa nonexpansive mapping, namely,Px−Py ≤ x−y.

It is well-known thatA I−P is inverse strongly monotone withL 2. Indeed, by

definition ofP, we have

_I₋_Ax₋_I₋_Ay_≤_x₋_y_. _3.13

On the other hand

_I₋_Ax₋_I₋_Ay2_x₋_y2_Ax₋_Ay2₋

(10)

Combining the two last inequalities, we obtain

x−y, Ax−Ay≥ 1

2Ax−Ay

2_. _3.15

Therefore, by Theorem2.1we get the weak convergence of the sequence xkgenerated by

the following algorithm:

xkproxβkΨI−λkxk−1λkPxk−1 3.16

to a solution of3.12provided that∞_k₁δkx<∞for allx∈VII−P, Candλk∈ε,1−ε.

The strong convergence of1.9is obtained, by applying Proposition2.3, forP a contraction

mapping, namely,Px−Py ≤γx−yfor 0< γ <1 which is equivalent to the1−γ-strong

monotonicity ofI−P, andλkis a convergent sequence withλk∈ε,21−γ/1γ2−ε.

It is easily seen that in this caseI−P is1γ-Lipschitz continuous.

4. Towards the Multivalued Case

Now, we are interested in1.2whenA : H → 2H is a multi-valued maximal monotone

operator. With the help of the Yosida approximate which is always inverse strongly monotone

and thus single-valued, we consider the following partial regularized version of1.2:

Aγx∗

γNC

x∗

γ

0, 4.1

whereAγstands for the Yosida approximate ofA.

It is well-known thatAγis inverse strongly monotone. More precisely, we have

Aγx−Aγy, x−y≥γAγx−Aγy2_. _4.2

Using definition of the Yosida approximate, algorithm1.9applied to4.1reads as

xγ_kIλkβk∂Ψ−1

1−λk

γ

x_k−γ ₁λk_γ JA

γ

xγ_k−₁. 4.3

From Theorem2.1, we infer thatxγ_k converges weakly to a solutionxγ provided that λk ∈

ε,2γ−ε. Furthermore, it is worth mentioning that ifA is strongly monotone,Aγ is also

strongly monotone, and thus4.1has a unique solutionxγ. By a result in8, page 35, we

have the following estimate:

_x₋_xγ_≤_o_γ_. _4.4

Consequently,4.3provides approximate solutions to the variational inclusion1.2for small

values ofγ. Furthermore, whenA∇Φ, we have

∂Φγx NCx ∇Φγx NCx ∂

(11)

and thus4.1reduces to

min

x∈CΦγx. 4.6

If3.1and4.1are solvable, by11Theorem 3.3, we have for allγ >0

0≤min

x∈C Φx−minx∈C Φγx≤γy

2_, _4.7

wherey ∇Φy∈ −NCxwith xa solution of3.1. The value of3.1is thus close to

those of4.1for small values ofγ, and hence, this confirmed the pertinence of the proposed

approximation idea to reach the multi-valued case. Observe that in this context, algorithm

4.3reads as

xγ_kproxβkΨ

1−λ_γk

xγ_k−₁ λ_γkproxγΦ

xγ_k−₁. 4.8

5. Conclusion

The authors have introduced a forward-backward penalization-gradient algorithm for solving variational inequalities and studied their asymptotic convergence properties. We have provided some applications to hierarchical fixed-point and optimization problems and also proposed an idea to reach monotone variational inclusions.

Acknowledgment

We gratefully acknowledge the constructive comments of the anonymous referees which helped them to improve the first version of this paper.

References

1 G. Stampacchia, “Formes bilin´eaires coercitives sur les ensembles convexes,” Comptes Rendus de

l’Acad´emie des Sciences de Paris, vol. 258, pp. 4413–4416, 1964.

2 H. Attouch, M. O. Czarnecki, and J. Peypouquet, “Prox-penalization and splitting methods for

constrained variational problems,” SIAM Journal on Optimization, vol. 21, no. 1, pp. 149–173, 2011.

3 G. B. Passty, “Ergodic convergence to a zero of the sum of monotone operators in Hilbert space,”

Journal of Mathematical Analysis and Applications, vol. 72, no. 2, pp. 383–390, 1979.

4 B. Lemaire, “Coupling optimization methods and variational convergence,” in Trends in Mathematical

Optimization, vol. 84 of International Series of Numerical Mathematics, pp. 163–179, Birkh¨auser, Basel, Switzerland, 1988.

5 J. Gwinner, “On the penalty method for constrained variational inequalities,” in Optimization: Theory

and Algorithms, vol. 86 of Lecture Notes in Pure and Applied Mathematics, pp. 197–211, Dekker, New York, NY, USA, 1983.

6 R. T. Rockafellar and R. J.-B. Wets, Variational Analysis, vol. 317, Springer, Berlin, Germany, 1998.

7 P. L. Martinet, Algorithmes pour la résolution de problèmes d’optimisation et de minimax, thesis, Université

de Grenoble, 1972.

8 H. Br´ezis, Op´erateurs Maximaux Monotones et Semi-Groupes de Contractions dans les Espaces de Hilbert,

(12)

9 J. M. Ortega and W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, NY, USA, 1970.

10 D. Pascali and S. Sburlan, Nonlinear Mappings of Monotone Type, Martinus Nijhoﬀ, The Hague, The

Netherlands, 1978.

11 N. Lehdili, Méthodes proximales de sélection et de décomposition pour les inclusions monotones, thesis,

(13)

Submit your manuscripts at

http://www.hindawi.com

Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

Mathematics

Journal of

Hindawi Publishing Corporation http://www.hindawi.com

Differential Equations

International Journal of

Volume 2014

Applied MathematicsJournal of

Mathematical PhysicsAdvances in

Complex Analysis

Journal of

Optimization

Journal of Hindawi Publishing Corporation

Combinatorics

International Journal of

Journal of

Function Spaces

Abstract and Applied Analysis Hindawi Publishing Corporation

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporation http://www.hindawi.com Volume 2014

The Scientific

World Journal

Discrete Dynamics in Nature and Society

Discrete Mathematics

Journal of

A Penalization Gradient Algorithm for Variational Inequalities

Research Article