PARALLEL SYNCHRONOUS ALGORITHM FOR
NONLINEAR FIXED POINT PROBLEMS
AHMED ADDOU AND ABDENASSER BENAHMED
Received 11 May 2004
We give in this paper a convergence result concerning parallel synchronous algorithm for nonlinear fixed point problems with respect to the Euclidian norm inRn. We then apply this result to some problems related to convex analysis like minimization of functionals, calculus of saddle point, and convex programming.
1. Introduction
This study is motivated by the paper of Bahi [2], where he has given a convergence result concerning parallel synchronous algorithm for linear fixed point problems using non-expansive linear mappings with respect to a weighted maximum norm. Our goal is to extend this result to a nonlinear fixed point problem
Fx∗=x∗, (1.1)
with respect to the Euclidian norm, whereF:Rn→Rnis a nonlinear operator.Section 2 is devoted to a brief description of asynchronous parallel algorithm. In Section 3, we prove the main result concerning the convergence of the general algorithm in the syn-chronous case to a fixed point of a nonlinear operator fromRntoRn. A particular case of this algorithm (algorithm of Jacobi) is applied inSection 4to the operatorF=(I+T)−1
which is called the proximal mapping associated with the maximal monotone operatorT (see Rockafellar [10]).
2. Preliminaries on asynchronous algorithms
Asynchronous algorithms are used in the parallel treatment of problems taking in con-sideration the interaction of several processors. WriteRnas the productα
i=1Rni, where
α∈N− {0} and n=iα=1ni. All vectors x∈Rn considered in this study are splitted
in the formx=(x1,...,xα), where xi∈Rni. Let Rni be equipped with the inner
prod-uct·,·i and the associated norm · i= ·,·1i/2.Rnwill be equipped with the inner
Copyright©2005 Hindawi Publishing Corporation
productx,y =αi=1xi,yii, wherex,y∈Rnand the associated normx = x,x1/2=
(αi=1xi2i)1/2. It will be equipped also with the maximum norm defined by
x∞=max1≤i≤αxii. (2.1)
DefineJ= {J(p)}p∈Nas a sequence of nonempty subsets of{1,...,α}andS= {(s1(p),...,
andsα(p))}p∈Nas a sequence ofNαsuch that
(i) for alli∈ {1,...,α}, the subset{p∈N,i∈J(p)}is infinite; (ii) for alli∈ {1,...,α}, for allp∈N,si(p)≤p;
(iii) for alli∈ {1,...,α}, limp→∞si(p)= ∞.
Consider an operatorF=(F1,...,Fα) :Rn→Rnand define the asynchronous algorithm
associated withFby
x0=x0 1,...,x0α
∈Rn,
xp+1
i =
xip ifi /∈J(p), Fi xs11(p),...,x
sα(p)
α
ifi∈J(p), i=1,...,α,
p=0, 1,...
(2.2)
(see Bahi and et al. [3], El Tarazi [4]). It will be denoted by (F,x0,J,S). This algorithm
de-scribes the behavior of iterative process executed asynchronously on a parallel computer withαprocessors. At each iterationp+ 1, theith processor computesxip+1by using (2.2) (see Bahi [1]).
J(p) is the subset of the indexes of the components updated at thepth step.p−si(p) is the delay due to theith processor when it computes theith block at thepth iteration.
If we takesi(p)=pfor alli∈ {1,...,α}, then (2.2) describes synchronous algorithm (without delay). During each iteration, every processor executes a number of computa-tions that depend on the results of the computacomputa-tions of other processors in the previous iteration. Within an iteration, each processor does not interact with other processors, all interactions take place at the end of iterations (see Bahi [2]).
If we take
si(p)=p ∀p∈N,∀i∈ {1,...,α},
J(p)= {1,...,α} ∀p∈N, (2.3)
then (2.2) describes the algorithm of Jacobi. If we take
si(p)=p ∀p∈N,∀i∈ {1,...,α},
J(p)=p+ 1 (modα) ∀p∈N, (2.4)
Theorem2.1. Consider{Tp}p∈N a sequence of matrices inRn×n. Suppose the following
hold.
(h0)There exists a subsequence{pk}k∈Nsuch thatJ(pk)= {1,...,α}.
(h1)There existsγ0(γ0means thatγi>0 ∀i∈ {1,...,α}), for all p∈N,Tp is
nonexpansive (a matriceA∈Rn×nis said to be nonexpansive with respect to the norm · if for all x∈Rn, Ax ≤ x. Ais said to be paracontracting if for allx∈Rn,x=Ax ⇐⇒ Ax<x), with respect to a weighted maximum norm
·∞,γdefined by
x∈Rn, x∞,γ=max 1≤i≤α
xii
γi . (2.5)
(h2){Tp}p∈Nconverges to a matrixQwhich is paracontracting with respect to the norm ·∞,γ.
(h3)For allp∈N,ᏺ(I−Q)⊆ᏺ(I−Tp)(ᏺdenotes the null space), then
(1)for allx0∈Rn, the sequence{xp}p
∈Nis convergent inRn;
(2) limp→∞xp=x∗∈ᏺ(I−Q).
For the proof, see Bahi [2].
Remark 2.2. The hypothesis (h0) means that the processors are synchronized and all the
components are infinitely updated at the same iteration. This subsequence can be chosen by the programmer.
3. Convergence of the general algorithm
We establish in this section the convergence of the general parallel synchronous algorithm to a fixed point of a nonlinear operatorF:Rn→Rnwith respect to the Euclidian norm defined inSection 2. We recall that an operatorFfromRntoRnis said to be nonexpansive with respect to a norm · if
∀x,y∈Rn, F(x)−F(y)≤ x−y. (3.1)
Theorem3.1. Suppose
(h0)there exists a subsequence{pk}k∈Nsuch thatJ(pk)= {1,...,α}; (h1)there existu∈Rn,F(u)=u;
(h2)for allx,y∈Rn,F(x)−F(y)∞≤ x−y∞;
(h3)for allx,y∈Rn,F(x)−F(y)2≤ F(x)−F(y),x−y.
Then any parallel synchronous (in this case,si(p)=p∀i∈ {1,...,α} ∀p∈N) algorithm defined by (2.2) associated with the operatorFconverges to a fixed pointx∗ofF.
Proof. (i) We prove first that the sequence{xp}p
∈Nis bounded. For alli∈ {1,...,α}, we
have eitheri /∈J(p) so
xp+1
ori∈J(p) so
xp+1
i −uii=Fixp−Fi(u)i
≤Fxp−F(u) ∞ ≤xp−u
∞
byh2
,
(3.3)
so
∀i∈ {1,...,α}, xip+1−uii≤xp−u∞, (3.4)
then
∀p∈N, xp+1−u
∞≤xp−u∞, (3.5)
hence
∀p∈N, xp−u
∞≤x0−u∞. (3.6)
This proves that the sequence{xp}p
∈Nis bounded with respect to the maximum norm
and then it is bounded with respect to the Euclidian norm. (ii) As the sequence{xpk}k
∈N is bounded ({pk}k∈N is defined by (h0)), it contains a
subsequence noted also as{xpk}k
∈Nwhich is convergent inRnto anx∗. We show thatx∗
is a fixed point ofF. For this, we consider the sequence{yp=xp−F(xp)}
p∈Nand prove
that limk→∞ypk=0,
xpk−u2
=ypk+Fxpk−u2 =ypk2
+Fxpk−u2
+ 2Fxpk−u,ypk, (3.7)
however
Fxpk−u,ypk=Fxpk−F(u),xpk−Fxpk
=Fxpk−F(u),xpk−F(u)−Fxpk−F(u)
=Fxpk−F(u),xpk−u−Fxpk−F(u)2 ≥0 byh3
,
(3.8)
so
ypk2
≤xpk−u2
−Fxpk−u2 =xpk−u2
−xpk+1−u2 byh 0
. (3.9)
However, in (i) we have shown in particular that the sequence{xp−u
∞}p∈Nis
decreas-ing (and it is positive), it is therefore convergent, then the sequence{xp−u}p∈Nis also
convergent, so
lim p→∞x
p−u=lim k→∞
xpk−u=lim k→∞
xpk+1−u=x∗−u,
and so
lim k→∞
ypk=0, (3.11)
which implies that
x∗−Fx∗=0, (3.12)
that is,x∗is a fixed point ofF.
(iii) We prove as in (i) that the sequence{xp−x∗∞}p∈Nis convergent, so
lim p→∞x
p−x∗
∞=klim→∞xpk−x∗∞=0, (3.13)
which proves thatxp→x∗with respect to the uniform norm··∞. Remark 3.2. We have used the hypothesis (h2) to prove that the sequence{xp}p∈N is
bounded. In the case of the parallel algorithm of Jacobi, whereJ(p)= {1,...,α}for all p∈N, we do not need this hypothesis, since in this casexp+1=F(xp) for allp∈N, and
use (h3) to obtain
xp+1−u=Fxp−F(u)≤xp−u, (3.14)
hence we have the following corollary.
Corollary3.3. Under the hypotheses(h1),(h3), and(h
0)for allp∈N,J(p)= {1,...,α}.
The parallel Jacobi algorithm defined by
x0=x0 1,...,x0α
∈Rn, xp+1
i =Fix1p,...,x
p α, i=1,...,α,
p=1, 2...
(3.15)
converges inRnto anx∗fixed point ofF.
4. Applications
4.1. Solutions of maximal monotone operators. In this section, we apply the parallel Jacobi algorithm to the proximal mappingF=(I+T)−1 associated with the maximal
monotone operatorT. We give first a general result concerning the maximal monotone operators. Such operators have been studied extensively because of their role in convex analysis (minimization of functionals, min-max problems, convex programming, etc.) and certain partial differential equations (see Rockafellar [10]).
Theorem4.1. LetT be a multivalued maximal monotone operator such thatT−10=φ.
Then every parallel Jacobi algorithm associated with the single-valued mappingF=(I+ T)−1converges inRnto anx∗solution of the problem0∈Tx.
Proof.
0∈Tx ⇐⇒ x∈(I+T)x
⇐⇒ x=(I+T)−1x ⇐⇒ x=Fx.
(4.1)
Thus, the solutions ofT are the fixed points ofF, so the condition T−10=φimplies
the existence of a fixed pointuofRn. It remains to show thatF verifies condition (h
3)
andCorollary 3.3. Consideringxi∈Rn(i=1, 2) and puttingyi=Fxi, thenxi∈yi+T yi orxi−yi∈T yi. As T is monotone, we have(x1−y1)−(x2−y2),y1−y2 ≥0, and
thereforex1−x2,y1−y2 − y1−y22≥0, which implies thatFx1−Fx22≤ Fx1−
Fx2,x1−x2.
4.2. Minimization of functional
Corollary4.2. Let f :Rn→R∪ {∞}be a lower semicontinuous convex function which is proper (i.e, not identically+∞). Suppose that the minimization problemminRn f(x)has a solution. Then any parallel Jacobi algorithm associated with the single-valued mapping F=(I+∂ f)−1converges to a minimizer of f inRn.
Proof. Since in this case the subdifferential∂ f is maximal monotone. However, the min-imizers of f are the solutions of∂ f; we then applyTheorem 4.1to∂ f.
4.3. Saddle point. In this paragraph, we applyTheorem 4.1to calculate a saddle point of functionalL:Rn×Rp→[−∞, +∞]. Recall that a saddle point ofLis an element (x∗,y∗)
ofRn×Rpsatisfying
Lx∗,y≤Lx∗,y∗≤Lx,y∗, ∀(x,y)∈Rn×Rp, (4.2)
which is equivalent to
Lx∗,y∗= inf x∈RnL
x,y∗=sup y∈RpL
x∗,y. (4.3)
Suppose thatL(x,y) is convex lower semicontinuous inx∈Rnand concave upper semi-continuous iny∈Rp. Such functionals are called saddle functions in the terminology of Rockafellar [6]. LetTLbe a multifunction defined inRn×Rpby
(u,v)∈TL(x,y)⇐⇒ L(x,y) +y−y,v ≤L(x,y)≤L(x,y)− x−x,u
∀(x,y)∈Rn×Rp. (4.4)
xand maximizing iny) are the elements (x,y) solutions of the problem (0, 0)∈TL(x,y). That is,
(0, 0)∈TLx∗,y∗ ⇐⇒ x∗,y∗=arg minx∈Rnmaxy∈RpL(x,y). (4.5)
We can then applyTheorem 4.1to the operatorTL, so we have the following corollary.
Corollary4.3. LetLbe a proper saddle function fromRn×Rpinto[−∞, +∞]having a saddle point. Then any parallel Jacobi algorithm associated with the single-valued mapping F=(I+TL)−1fromRn×RpintoRn×Rpconverges to a saddle point ofL.
4.4. Convex programming. We consider now the convex programming problem: (P)
minf0(x), x∈Rn,
fi(x)≤0 (1≤i≤m), (4.6)
where fi are lower semicontinuous convex functions. This problem can be re-duced to an unconstrained one by means of the Lagrangian
L(x,y)= f0(x) +
m
i=1
yifi(x), (4.7)
wherex∈Rnand y∈(R+)m. We observe thatLis a saddle function in the sense of [6,
page 363], due to the assumptions of convexity and continuity. The dual problem associ-ated with (P) is
(D)
maxg0(y)=xinf ∈RnL(x,y)
y∈R+
m
. (4.8)
If (x∗,y∗) is a saddle point of the LagrangianL, thenx∗is an optimal solution of the
primal problem (P) andy∗is an optimal solution of the dual problem (D).
Let∂L(x,y) the subdifferential ofLat (x,y)∈Rn×Rpbe defined as the set of vectors (u,v)∈Rn×Rpsatisfying
∀(x,y)∈Rn×Rp, L(x,y)− y−y,v ≤L(x,y)≤L(x,y)− x−x,u (4.9)
(see Luque [5] and Rockafellar [6]). Then the operator TL: (x,y)→ {(u,v) : (u,−v)∈ ∂L(x,y)} is maximal monotone (see Rockafellar [6, Corollary 37.5.2]), so we apply
Theorem 4.1toTL.
4.5. Variational inequality. A simple formulation of the variational inequality problem is to find anx∗∈Rnsatisfying
Ax∗,x−x∗≥0 ∀x∈Rn, (4.10)
whereA:Rn→Rnis a single-valued monotone and maximal operator. (In fact, it is suf-ficient that A is monotone and hemicontinuous, i.e, verifying limt→0+A(x+ty),h = Ax,h ∀x,y,h∈Rn.) This is equivalent to finding anx∗∈Rnsuch that
0∈Ax∗+Nx∗, (4.11)
whereN(x) is the normal cone toRnatxdefined by (see Rockafellar [6,10])
N(x)=y∈Rn:y,x−z ≥0∀z∈Rn. (4.12)
Rockafellar [10] has considered the multifunctionTdefined inRnby
Tx=Ax+N(x) (4.13)
and shown in [8] that T is maximal monotone. The relation 0∈Tx∗ is reduced to −Ax∗∈N(x∗) or −Ax∗,x∗−z ≥0 for all z∈Rnwhich is the variational inequal-ity (4.10). Therefore, the solutions of the operatorT (defined by (4.13)) are exactly the solutions of the variational inequality (4.10). By using Theorem 4.1, we can write the following corollary.
Corollary4.5. LetA:Rn→Rnbe a single-valued monotone and hemicontinuous opera-tor such that the problem (4.10) has a solution, then any parallel Jacobi algorithm associated with the single-valued mappingF=(I+T)−1, whereTis defined by (4.13), converges tox∗ solution of the problem (4.10).
References
[1] J. Bahi,Algorithmes parall`eles asynchrones pour des syst`emes singuliers[Asynchronous parallel algorithms for a class of singular linear systems], C. R. Acad. Sci. Paris S´er. I Math.326(1998), no. 12, 1421–1425.
[2] , Parallel chaotic algorithms for singular linear systems, Parallel Algorithms Appl.14
(1999), no. 1, 19–35.
[3] J. Bahi, J. C. Miellou, and K. Rhofir,Asynchronous multisplitting methods for nonlinear fixed point problems, Numer. Algorithms15(1997), no. 3-4, 315–345.
[4] M. N. El Tarazi,Some convergence results for asynchronous algorithms, Numer. Math.39(1982), no. 3, 325–340 (French).
[5] F. J. Luque,Asymptotic convergence analysis of the proximal point algorithm, SIAM J. Control Optim.22(1984), no. 2, 277–293.
[6] R. T. Rockafellar,Convex Analysis, Princeton Mathematical Series, no. 28, Princeton University Press, New Jersey, 1970.
[7] ,Monotone operators associated with saddle-functions and minimax problems, Nonlinear Functional Analysis (Proc. Sympos. Pure Math., Vol. 18, part 1, Chicago, Ill, 1968), Ameri-can Mathematical Society, Rhode Island, 1970, pp. 241–250.
[8] ,On the maximality of sums of nonlinear monotone operators, Trans. Amer. Math. Soc.
[9] ,Augmented Lagrangians and applications of the proximal point algorithm in convex pro-gramming, Math. Oper. Res.1(1976), no. 2, 97–116.
[10] ,Monotone operators and the proximal point algorithm, SIAM J. Control Optimization
14(1976), no. 5, 877–898.
Ahmed Addou: D´epartement de Math´ematiques et Informatique, Facult´e des Sciences Oujda, Universit´e Mohammed Premier Oujda, 60 000 Oujda, Morocco
E-mail address:[email protected]
Abdenasser Benahmed: D´epartement de Math´ematiques et Informatique, Facult´e des Sciences Oujda, Universit´e Mohammed Premier Oujda, 60 000 Oujda, Morocco
