R E S E A R C H
Open Access
New descent LQP alternating direction
methods for solving a class of structured
variational inequalities
Abdellah Bnouhachem
1,2, Suliman Al-Homidan
3*and Qamrul Hasan Ansari
3,4*Correspondence:
[email protected] 3Department of Mathematics &
Statistics, King Fahd University of Petroleum & Minerals, Dhahran, Saudi Arabia
Full list of author information is available at the end of the article
Abstract
In this paper, we propose two descent alternating direction methods based on a logarithmic-quadratic proximal method for structured variational inequalities. The first method can be viewed as an extension of the method proposed by Bnouhachem and Xu (Comput. Math. Appl. 67:671-680, 2014) by performing an additional step at each iteration. The second method generates the new iterate by searching the optimal step size along a new descent direction, which can be viewed as a refinement and improvement of the first one. Under certain conditions, the global convergence of the both methods is proved.
MSC: Primary 49J30; 47H09; secondary 47J20; 49J40
Keywords: structured variational inequalities; logarithmic-quadratic proximal method; projection method; alternating direction method
1 Introduction
We consider the constrained convex programming problem with the following separate structure:
minθ(x) +θ(y)|Ax+By=b,x∈Rn+,y∈Rm+
, (.)
whereθ:Rn+→Randθ:Rm+ →Rare closed proper convex functions,A∈Rl×n,
B∈Rl×mare given matrices, andb∈Rlis a given vector.
A large number of problems can be modeled as problem (.). In practice, these classes of problems have very large size and, due to their practical importance, they have received a great deal of attention from many researchers. Various methods have been suggested to find the solution of problem (.). A popular approach is the alternating direction method (ADM) which was proposed by Gabay and Mercier [] and Gabay []. The ADM can re-duce the scale of variational inequalities by decomposing the original problem into a series of subproblems with a lower scale. To make the ADM more efficient and practical, some strategies have been studied; For further details, we refer to [–] and the references therein.
Let∂(·) denote the sub-gradient operator of a convex function, andf(x)∈∂θ(x) and
g(y)∈∂θ(y) are the sub-gradient ofθ(x) andθ(y), respectively. By attaching a Lagrange
multiplier vectorλ∈Rlto the linear constraintAx+By=b, problem (.) can be written in terms of findingw∈Wsuch that
w–wQ(w)≥, ∀w∈W, (.)
where
w= ⎛ ⎜ ⎝
x y λ
⎞ ⎟
⎠, Q(w) = ⎛ ⎜ ⎝
f(x) –Aλ g(y) –Bλ Ax+By–b
⎞ ⎟
⎠, W=Rn+×Rm+ ×Rl. (.)
Problem (.)-(.) is referred to asstructured variational inequalities(in short, SVI). Very recently, Yuan and Li [] developed the following logarithmic-quadratic proximal (LQP)-based decomposition method by applying the LQP terms to regularize the ADM subproblems: For a givenwk= (xk,yk,λk)∈Rn++×Rm++×Rl, andμ∈(, ), the new iter-ative (xk+,yk+,λk+) is obtained via solving the following system:
f(x) –Aλk–HAx+Byk–b+Rx–xk+μxk–X
kx–
= , (.)
g(y) –Bλk–H(Ax+By–b)+Sy–yk+μyk–Yky–= , (.)
λk+=λk–HAxk+Byk–b, (.) whereH∈Rl×l,R∈Rn×n, andS∈Rm×mare symmetric positive definite.
Note that the LQP method was presented originally in []. It seems that it is easier to solve a series of systems of nonlinear equations than to solve a series of sub-variational inequalities in many cases. Later, Bnouhachemet al.[], Bnouhachem and Ansari [], and Li [] proposed some LQP alternating direction methods and made the LQP alter-nating direction method more practical. Each iteration of the above methods contains a prediction and a correction. The predictor is obtained via solving (.)-(.) and the new iterate is obtained by a convex combination of the previous point and the one generated by a projection-type method along a descent direction for [, ]. In , Bnouhachem and Xu [] proposed the following LQP alternating direction: For a givenwk= (xk,yk,λk)∈ Rn
++×Rm++×Rl, andμ∈(, ), the predictorw˜k= (x˜k,y˜k,λ˜k)∈R++n ×Rm++×Rlis obtained
via solving the following system:
f(x) –Aλk–H(Ax+By–b)+Rx–xk+μxk–Xkx–=:ξxk≈, (.a)
g(y) –Bλk–H(Ax+By–b)+Sy–yk+μyk–Yky–=:ξyk≈, (.b) ˜
λk=λk–HAx˜k+B˜yk–b, (.c)
where
G–ξk
G≤ –μ
+μη
wk–w˜k
G, η∈(, ), (.)
ξk= ⎛ ⎜ ⎝
ξxk ξyk
⎞ ⎟
and
G= ⎛ ⎜ ⎝
( +μ)R
( +μ)S H–
⎞ ⎟
⎠ (.)
is a positive definite (block diagonal) matrix.
Take the new iteratewk+= (xk+,yk+,λk+) as the solution of the following system:
τfx˜k–Aλ˜k+Rx–xk+μxk–Xkx–= , (.a)
τgy˜k–Bλ˜k+Sy–yk+μyk–Yky–= , (.b)
λk+=λk–τHAx˜k+By˜k–b. (.c)
Each iteration of the above method contains a prediction and a correction. The predictor is obtained via solving (.)-(.) and the new iterate is computed directly by an explicit formula derived from the original LQP method.
Inspired and motivated by the above research, we present two methods to solve SVI. The first one can be viewed as an extension of the method proposed in [] by performing an additional step at each iteration. The second method can be viewed as an extension of the first one by using a new descent direction which provides a significant refinement and improvement of the first one. We also study the global convergence of the both methods method under certain conditions.
2 Iterative methods and convergence results
In this section, we suggest and analyze two new modified logarithmic-quadratic proximal alternating direction methods for solving structured variational inequalities. The follow-ing lemma provides some basic properties of the projection onto.
Lemma . Let G be a symmetry positive definite matrix and be a nonempty closed convex subset of Rl.We denote by P
,G(·)the projection under the G-norm,that is,
P,G(v) =argmin
v–uG:u∈
.
Then
z–P,G[z]
GP,G[z] –v
≥, ∀z∈Rl,v∈; (.) P,G[u] –P,G[v]G≤ u–vG, ∀u,v∈Rl; (.) u–P,G[z]G≤ z–uG–z–P,G[z]G, ∀z∈Rl,u∈. (.)
For convenience, we make the following standard assumptions to guarantee that the problem under consideration is solvable and the proposed methods are well defined.
Assumption A f(x) is monotone with respect toRn
++andg(y) is monotone with respect
Assumption B The solution set of SVI, denoted byW∗, is nonempty.
Then the iterative scheme of the first method is given as follows.
Algorithm .
Step . The initial step:
Givenε> ,μ∈(, )andw= (x,y,λ)∈Rn++×Rm++×Rl. Setk= . Step . Prediction step:
Computew˜k= (x˜k,˜yk,λ˜k)∈Rn++×Rm++×Rlby solving the system (.a)-(.c). Step . Convergence verification:
If{maxxk–x˜k∞,yk–y˜k∞,λk–λ˜k∞}<, then stop.
Step . Correction step:
Computew¯k= (x¯k,¯yk,λ¯k)by solving the following system:
–μ
+μαk
fx˜k–Aλ˜k+Rx–xk+μxk–Xkx–= , (.a) –μ
+μαk
gy˜k–Bλ˜k+Sy–yk+μyk–Yky–= , (.b)
λk+=λk– –μ +μαkH
Ax˜k+By˜k–b, (.c)
where
αk=
ϕk dkG
, (.)
ϕk:=xk–x˜k
R+y k–y˜k
S+λ
k–λk+
H–+
wk–w˜kξk, (.)
and
dk:=wk–w˜k+G–ξk. (.)
The new iteratewk+(τ
k) = (xk+,yk+,λk+)is given by
wk+(τk) =ρwk+ ( –ρ)PWwk–τkdk
, ρ∈(, ), (.)
where
dk:=wk–w¯k. (.)
Setk:=k+ and go to Step . τkis a positive scalar.
How to choose a suitable step lengthτkto force convergence will be discussed later.
Theorem .[] For given wk= (xk,yk,λk)∈Rn++×Rm++×Rl,letw¯k= (x¯k,¯yk,λ¯k)be gen-erated by(.a)-(.c).Then for any w∗= (x∗,y∗,λ∗)∈W∗,we have
wk–w∗G–w¯k–w∗G≥ –μ
where
(αk) := αkϕk–αkdkG. (.) Theorem .[] For given wk∈Rn++×Rm++×Rl,letw˜kbe generated by(.b)-(.c),then we have the following:
αk≥
(.)
and
(αk)≥( –η
)( –μ)
( +μ) w k–w˜k
G. (.)
How should one choose values ofτkto ensure thatwk+(τk) is closer to the solution set thanwk? For this purpose, we define
(τk) =wk–w∗
G–w k+(τ
k) –w∗
G. (.)
Theorem . Let w∗= (x∗,y∗,λ∗)∈W∗,then we have
(τk)≥( –ρ)
τkdk
G+w k–w∗
G–w¯ k–w∗
G
–τkdkG. (.)
Proof Sincew∗= (x∗,y∗,λ∗)∈W∗andwk
∗(τk) =PW[wk–τkdk], it follows from (.) that wk∗(τk) –w∗G≤wk–τkdk–w∗
G–w k–τ
kdk–wk∗(τk)
G. (.)
On the other hand, we have
wk+(τk) –w∗
G=ρ
wk–w∗+ ( –ρ)wk∗(τk) –w∗
G
=ρwk–w∗G+ ( –ρ)wk∗(τk) –w∗G
+ ρ( –ρ)wk–w∗Gwk∗(τk) –w∗.
Using the identity
(a+b)Gb=a+bG –aG+bG,
fora=wk–wk
∗(τk),b=wk∗(τk) –w∗, and (.), we obtain
wk+(τk) –w∗G =ρwk–w∗G+ ( –ρ)wk∗(τk) –w∗G
+ρ( –ρ)wk–w∗G–wk–wk∗(τk)+wk∗(τk) –w∗
G
=ρwk–w∗G+ ( –ρ)wk∗(τk) –w∗
G–ρ( –ρ)w k–wk
∗(τk)
G
≤ρwk–w∗G+ ( –ρ)wk–τkdk–w∗
G
– ( –ρ)wk–τkdk–wk∗(τk)
G–ρ( –ρ)w k–wk
=wk–w∗G– ( –ρ)wk–wk∗(τk) –τkdk
G+ρw k–wk
∗(τk)G –τkdkG+ τk
wk–w∗Gdk
≤wk–w∗G– ( –ρ)τk
wk–w∗Gdk–τkdkG.
Using the definition of(τk), we get
(τk)≥( –ρ)
τk
wk–w∗Gdk–τkdkG
= ( –ρ)τkdk
G–
w∗–w¯kGdk–τkdkG. (.)
The identity
w∗–w¯kGdk= w¯
k–w∗
G–w k–w∗
G
+ d
k
G (.)
implies
dkG– w∗–w¯kGdk=wk–w∗G–w¯k–w∗G. (.)
Substituting (.) in (.), we get the assertion of this lemma.
Using Theorem . and Theorem ., we get
(τk)≥( –ρ)(τk), (.)
where
(τk) =τk
dkG+
–μ
+μ
(αk)
–τkdkG. (.)
(τk) measures the progress obtained in thekth iteration. It is natural to choose a step lengthτkwhich maximizes the progress. Note that(τk) is a quadratic function ofτkand it reaches its maximum at
τk∗=d
kG+ (
–μ
+μ)(αk) dkG
and
τk∗=τ
∗
k{dkG+ (
–μ
+μ)(αk)}
. (.)
Using (.), we have
τk∗≥( –μ)
( –η)
( +μ)
dkG+wk–w˜kG
dkG
≥( –μ)( –η)
which implies
τk∗≥
τk∗(–+μμ)(αk)
≥( –μ)( –η) ( +μ) w
k–w˜k
G. (.)
Remark . Ifρ= andτk∗= , Algorithm . reduces to the method proposed in []. Sinceτk∗is to maximize the profit function(τ), we have
τk∗≥(). (.)
Inequalities (.) and (.) show theoretically that Algorithm . is expected to make more progress than the method proposed in [] at each iteration, and so it explains theo-retically that Algorithm . outperforms the method proposed in [].
Inspired and motivated by the first method, we next propose the following new LQP alternating direction method for solving SVI.
Algorithm .
Step . The initial step:
Givenε> ,μ∈(, ), andw= (x,y,λ)∈Rn
++×Rm++×Rlandd= (, , ).
Setk= . Step . Prediction step:
Computew˜k= (x˜k,y˜k,λ˜k)∈Rn
++×Rm++×Rlby solving the system (.a)-(.c).
Step . Convergence verification:
If{maxxk–x˜k∞,yk–y˜k∞,λk–λ˜k∞}<, then stop. Step . Correction step:
Computew¯k= (x¯k,¯yk,λ¯k)by solving the system (.a)-(.c). The new iterate
wk+(β
k)is given by
wk+(βk) =ρwk+ ( –ρ)PWwk–βkdk
, ρ∈(, ), (.)
where
dk:=dk+σkdk–, (.)
σk=max
,–d
k Gdk–
dk–
G
,
and
βk= dk
G+ (
–μ
+μ)(αk)
dkG . (.)
Setk:=k+ and go to Step .
Lemma . For any k≥,we have
dk–Gwk–w∗≥. (.)
Proof From (.) and (.) it is easy to show that
wk–w∗Gdk≥d
kG+ (– μ
+μ)(αk)
≥( –μ)( –η) ( +μ) w
k–w˜k
G. (.)
Note that (.) is trivially true fork= sinced= (, , ). By induction, consider any
k≥ and assume that
dk–Gwk––w∗≥.
Using the definition ofdk–, we have
dk–Gwk–w∗=dk–Gwk––w∗+dk–Gwk–wk–
=dk–Gwk––w∗+σk–dk–G
wk––w∗
+dk–Gwk–wk–
≥dk–Gwk––w∗+dk–Gwk–wk–
≥ d
k–G+ (– μ
+μ)(αk–)
–dk–GPWwk––βk–dk–
–wk–G
≥ d
k–G+ (
–μ
+μ)(αk–)
–βk–d
k–
G
= ,
where the second inequality follows from (.). Hence, the lemma is proved.
Remark . From (.) and Lemma ., we get
wk–w∗Gdk≥( –μ)
( –η)
( +μ) w
k–w˜k G.
Then –dkis a descent direction of w
k–w∗ at the pointwk. Since –d
k is a descent direction of the distance function atwk, along –d
k, one can find a new iterate which is closer to the solution set. Due to this fact, we construct (.).
Lemma . Let w∗= (x∗,y∗,λ∗)∈W∗,then we have
(βk)≥( –ρ)
βkdk
G+w k–w∗
G–w¯ k–w∗
G
where
(βk) =wk–w∗
G–w k+(β
k) –w∗
G. (.)
Proof Similar to (.), we have
wk+(βk) –w∗G≤wk–w∗G– ( –ρ)βk
wk–w∗Gdk–βkdkG
=wk–w∗G– ( –ρ)βk
wk–w∗Gdk+σkdk–
–βkdkG
≤wk–w∗G– ( –ρ)βk
wk–w∗Gdk–βkdkG.
Using (.) and the definition of(βk), we get the assertion of this lemma.
Using Theorem . and Lemma ., we get
(βk)≥( –ρ)(βk), (.)
where
(βk) =βk
dkG+
–μ
+μ
(αk)
–βkdkG
=βk{d
kG+ (
–μ
+μ)(αk)}
. (.)
Remark . From (.), it is easy to prove that
dkG≤dkG,
which implies that
τk∗≤βk.
From (.), (.), (.), and (.), we have
τk∗≥( –ρ)
τk∗=τ
∗
k( –ρ){dkG+ (
–μ
+μ)(αk)}
and
(βk)≥( –ρ)(βk) =
βk( –ρ){dkG+ (– μ
+μ)(αk)}
.
From the above inequalities, we obtain
(βk)≥
τk∗. (.)
Now, we mainly focus on investigating the convergence of the proposed methods. The following theorem plays a crucial role in the convergence of the proposed methods.
Theorem . Let wk+be the new iterate generated by
wk+=wk+τk∗ or wk+=wk+(βk).
Then,for any w∗∈W∗,wkandw˜kare bounded,and
wk+–w∗≤wk–w∗–cwk–w˜kG, (.)
where
c:=( –μ)
( –η)
( +μ) > .
Proof Inequality (.) follows from (.), (.), (.), and (.) immediately. It fol-lows from (.) that
wk+–w∗≤wk–w∗≤ · · · ≤w–w∗,
and thus,{wk}is a bounded sequence. It follows from (.) that
∞
k=
cwk–w˜kG< +∞,
which means that
lim k→∞
wk–w˜kG= .
Since{wk}is a bounded sequence, we conclude that{ ˜wk}is also bounded.
The convergence of the proposed method can be proved by using similar arguments to []. Hence the proof is omitted.
Theorem .[] The sequence{wk}generated by the proposed methods converges to some
w∞which is a solution of SVI.
3 Conclusions
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
All authors contributed equally and significantly in writing this article. All authors read and approved the final manuscript.
Author details
1School of Management Science and Engineering, Nanjing University, Nanjing, 210093, P.R. China.2ENSA, Ibn Zohr
University, BP 1136, Agadir, Morocco.3Department of Mathematics & Statistics, King Fahd University of Petroleum & Minerals, Dhahran, Saudi Arabia.4Department of Mathematics, Aligarh Muslim University, Aligarh, 202002, India.
Acknowledgements
The authors are grateful to King Fahd University of Petroleum & Minerals for providing excellent research facilities.
Received: 12 May 2015 Accepted: 22 July 2015
References
1. Bnouhachem, A, Xu, MH: An inexact LQP alternating direction method for solving a class of structured variational inequalities. Comput. Appl. Math.67, 671-680 (2014)
2. Gabay, D, Mercier, B: A dual algorithm for the solution of nonlinear variational problems via finite-element approximations. Comput. Appl. Math.2, 17-40 (1976)
3. Gabay, D: Applications of the method of multipliers to variational inequalities. In: Fortin, M, Glowinski, R (eds.) Augmented Lagrange Methods: Applications to the Solution of Boundary-Valued Problems, pp. 299-331. North-Holland, Amsterdam (1983)
4. Chen, G, Teboulle, M: A proximal-based decomposition method for convex minimization problems. Math. Program.
64, 81-101 (1994)
5. Eckstein, J: Some saddle-function splitting methods for convex programming. Optim. Methods Softw.4, 75-83 (1994) 6. Kontogiorgis, S, Meyer, RR: A variable-penalty alternating directions method for convex optimization. Math. Program.
83, 29-53 (1998)
7. He, BS, Yang, H, Wang, SL: Alternating directions method with self-adaptive penalty parameters for monotone variational inequalities. J. Optim. Theory Appl.106, 349-368 (2000)
8. He, BS, Zhou, J: A modified alternating direction method for convex minimization problems. Appl. Math. Lett.13, 123-130 (2000)
9. He, BS, Liao, LZ, Han, DR, Yang, H: A new inexact alternating directions method for monotone variational inequalities. Math. Program.92, 103-118 (2002)
10. Jiang, ZK, Bnouhachem, A: A projection-based prediction-correction method for structured monotone variational inequalities. Appl. Math. Comput.202, 747-759 (2008)
11. Yang, J, Zang, Y: Alternating direction algorithms forl1problems in compressive sensing. SIAM J. Sci. Comput.33(1), 250-278 (2011)
12. Tao, M, Yuan, XM: On theO(1/t) convergence rate of alternating direction method with logarithmic-quadratic proximal regularization. SIAM J. Optim.22(4), 1431-1448 (2012)
13. Wang, K, Xu, LL, Han, DR: A new parallel splitting descent method for structured variational inequalities. J. Ind. Manag. Optim.10(2), 461-476 (2014)
14. Yuan, XM, Li, M: An LQP-based decomposition method for solving a class of variational inequalities. SIAM J. Optim.
21(4), 1309-1318 (2011)
15. Auslender, A, Teboulle, M, Ben-Tiba, S: A logarithmic-quadratic proximal method for variational inequalities. Comput. Optim. Appl.12, 31-40 (1999)
16. Bnouhachem, A, Benazza, H, Khalfaoui, M: An inexact alternating direction method for solving a class of structured variational inequalities. Appl. Math. Comput.219, 7837-7846 (2013)
17. Bnouhachem, A, Ansari, QH: A descent LQP alternating direction method for solving variational inequality problems with separable structure. Appl. Math. Comput.246, 519-532 (2014)
