A Note on Convergence Analysis of an SQP Type Method for Nonlinear Semidefinite Programming

Volume 2008, Article ID 218345,10pages doi:10.1155/2008/218345

Research Article

A Note on Convergence Analysis of an SQP-Type

Method for Nonlinear Semidefinite Programming

Yun Wang,1 _{Shaowu Zhang,}2 _{and Liwei Zhang}1

1_{Department of Applied Mathematics, Dalian University of Technology, Dalian 116024, China} 2_{Department of Computer Science, Dalian University of Technology, Dalian 116024, China}

Correspondence should be addressed to Yun Wang,wangyun [email protected]

Received 29 August 2007; Accepted 23 November 2007

Recommended by Kok Lay Teo

We reinvestigate the convergence properties of the SQP-type method for solving nonlinear semidef-inite programming problems studied by Correa and Ramirez2004. We prove, under the strong second-order sufficient condition with the sigma term, that the local SQP-type method is quadrati-cally convergent and the line search SQP-type method is globally convergent.

Copyrightq2008 Yun Wang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

We consider the following nonlinear semidefinite programming:

SDP minfx s.t. hx 0, gx∈ Sp, 1.1

where x ∈ Rn,f : Rn→R, h : Rn→Rl, andg : Rn→Sp are twice continuously differentiable functions,Sp is the linear space of allp×preal symmetric matrices, andSp is the cone of all

p×psymmetric positive semidefinite matrices.

Comparing with the work by Correa and Ramirez2004 2 , in this note, we make some modifications to the convergence analysis, and prove that all results in2 still hold under the strong second-order sufficient condition with the sigma term.

It should be pointed out that the importance of exploring numerical methods for solving nonlinear semidefinite programming problems has been recognized in the optimization com-munity. For instance, Koˇcvara and Stingl3,4 have developed PENNLP and PENBMI codes for nonlinear semidefinite programming and semidefinite programming with bilinear matrix inequality constraints, respectively. Recently, Sun et al.2007 5 considered the rate of con-vergence of the classical augmented Lagrangian method and Noll2007 6 investigated the convergence properties of a class of nonlinear Lagrangian methods.

In Section 2, we introduce preliminaries including differential properties of the metric projector ontoSpand optimality conditions for problem1.1. InSection 3, we prove, under the strong second-order sufficient condition with the sigma term, that the local SQP-type method has the quadratic convergence rate and the global algorithm with line search is convergence.

2. Preliminaries

We use Rm×n to denote the set of all the matrices ofmrows and ncolumns. For Aand Bin

Rm×n_{, we use the Frobenius inner productA, BtrA}T_{B, and the Frobenius normA} F

trAT_{A, where “tr” denotes the trace operation of a square matrix.} For a given matrixA∈ Sp_{, its spectral decomposition is}

APΛPT P ⎛ ⎜ ⎝

λ1 0 0 0 . .. 0 0 0 λp

⎞ ⎟

⎠PT, 2.1

whereΛis the diagonal matrix of eigenvalues ofAandPis a corresponding orthogonal matrix. We can expressΛandP as

Λ

⎛

⎝Λα_{0 0 0}0 0

0 0 Λγ

⎞

⎠_{, P Pα Pβ Pγ, 2.2}

whereα,β,γ are the index sets of positive, zero, negative eigenvalues ofA, respectively.

2.1. Semismoothness of the metric projector

In this subsection, letX,Y, andZbe three arbitrary finite-dimensional real spaces with a scalar product·,·and its norm·. We introduce some properties of the metric projector, especially its strong semismoothness.

The next lemma is about the generalized Jacobian for composite functions, proposed in 7 .

Lemma 2.1. LetΨ : X→Y be a continuously differentiable function on an open neighborhoodNof

y : Ψx. Suppose thatΞis directionally differentiable at every point inOand thatJxΨx:X→Y

is onto. Then it holds that

∂BΦx ∂BΞyJxΨx, 2.3

whereΦ:N→Zis defined byΦx: ΞΨx,x∈N.

The following concept of semismoothness was first introduced by Mifflin8 for func-tionals and was extended by Qi and Sun in9 to vector valued functions.

Definition 2.2. LetΦ: O ⊆X→Y be a locally Lipschitz continuous function on the open setO. One says thatΦis semismooth at a pointx∈ Oif

i Φis directionally differentiable atx;

iifor anyΔx∈XandV ∈∂Φx ΔxwithΔx→0,

Φx Δx−Φx−VΔx oΔx. 2.4

Furthermore,Φis said to be strongly semismooth atx ∈ O ifΦis semismooth atxand for anyΔx∈XandV ∈∂Φx ΔxwithΔx→0,

Φx Δx−Φx−VΔx OΔx2. 2.5

Let D be a closed convex set in a Banach space Z, and let ΠD : Z→Z be the metric projector overD. It is well known in10,11 thatΠD·isF-differentiable almost everywhere

inZand for anyy∈Z,∂ΠDyis well defined.

SupposeA∈ Sp, then it has the spectral decomposition as2.1, then the merit projector ofAtoSpis denoted byΠ_Sp

Aand

Π_Sp

A P

⎛ ⎜ ⎝

λ1

0 0

0 . .. 0 0 0 λp

⎞ ⎟

⎠PT, 2.6

whereλi max{0, λi}, i1, . . . , p.

For our discussion, we know from12 that the projection operatorΠ_Sp

·is directionally

differentiable everywhere inSp

and is a strongly semismooth matrix-valued function. In fact, for anyA∈ Sp_{,H ∈ S}p

, there existsV ∈∂ΠSp

AH, satisfying

Π_Sp

AH ΠSpA VH O

H2. 2.7

2.2. Optimality conditions

Let the Lagrangian function of1.1be

Robinson’s constraint qualificationCQis said to hold at a feasible pointxif

0∈int

hx gx

Jhx

Jgx

Rn₋

{0}

Sp

. 2.9

Ifxis a locally optimal solution to1.1and Robinson’s CQ holds atx, then there exist Lagrangian multipliersλ, μ∈Rl_{× S}p_{, such that the following KKT condition holds:}

0∇xLx, λ, μ ∇fx Jhx∗λJgx∗μ, 0hx,

gx Π_Sp

gx μ, 2.10

which is equivalent toFx, λ, μ 0, where

Fx, λ, μ: ⎛ ⎜ ⎝

∇fx Jhx∗λJgx∗μ hx

gx−Π_Sp

gx μ ⎞ ⎟

⎠. 2.11

Let Λx be the set of all the Lagrangian multipliers satisfying 2.10. Then Λx is a nonempty, compact convex set of Rl _{× S}p _{if and only if Robinson’s CQ holds at x, see13 .} Moreover, it follows from13 that the constraint nondegeneracy condition is a sufficient con-dition for Robinson constraint qualification. In the setting of the problem1.1, the constraint nondegeneracy condition holding at a feasible pointxcan be expressed as

Jhx

Jgx

Rn

{0} linT_Sp

gx

Rl Sp , 2.12

where linT_Sp

gx is the lineality space of the tangent cone of S

p

at gx. If x, a locally optimal solution to1.1, is nondegenerate, thenΛxis a singleton.

For a KKT pointx, λ, μof1.1, without loss of generality, we assume thatgxandμ

have the spectral decomposition forms

gx P ⎛

⎝Λα_{0 0 0}0 0

0 0 0

⎞

⎠_PT_{, μP}

⎛ ⎝0 0 0_{0 0 0}

0 0 Λγ

⎞

⎠_PT_{. 2.13}

We state the strong second-order sufficient conditionSSOSCcoming from7 .

Definition 2.3. Letxbe a stationary point of1.1such that2.12holds atx. One says that the strong second-order sufficient condition holds atxif

d,∇2_xxLx, λ, μd−Υgxμ,Jgxd>0, ∀d∈affCx\ {0}, 2.14

where{λ, μ} Λx⊂Rl_{× S}p_{, affCxis the affine hull of the critical coneCx:} affCx d:Jhxd0, P_βTJgxdPγ 0, PγT

JgxdPγ 0

. 2.15

And the linear-quadratic functionΥB :Sp_{× S}p_{→Ris defined by}

ΥBD, A:2D, AB†A, D, A∈ Sp× Sp, 2.16

The next proposition relates the SSOSC and nondegeneracy condition to nonsingularity of Clarke’s Jacobian of the mappingFdefined by2.11. The details of this proof can be found in7 .

Proposition 2.4. Letx, λ, μbe a KKT point of 1.1. If nondegeneracy condition2.12and SSOSC

2.14hold atx, then any element in∂Fx, λ, μis nonsingular, whereFis defined by2.11.

3. Convergence analysis of the SQP-type method

In this section, we analyze the local quadratic convergence rate of an SQP-type method and then prove that the SQP-type method proposed in 2 is globally convergent. The analysis is based on the strong second-order sufficient condition, which is weaker than the conditions used in1,2 .

3.1. Local convergence rate

Linearizing 1.1 at the current point xk, λk, μk, we obtain the following tangent quadratic

problem:

minΔx∇fxkTΔx1₂ΔxT∇2xxL

xk, λk, μkΔx,

s.t. hxkJhxkΔx0, gxkJgxkΔx∈ Sp,

3.1

where ∇2xxLxk, λk, μk Jx∇xLxk, λk, μk. Let Δxk, λkQP, μkQPbe a KKT point of 3.1, then we haveFΔxk_{, λ}k

QP, μkQP;xk, λ

k_{, μ}k _{0, where}

Fζ, η, ξ;xk, λk, μk: ⎛ ⎜ ⎝∇f

xk∇2xxL

xk, λk, μkζJhxk∗ηJgxk∗ξ hxk Jhxkζ

gxk_Jgxk_ζ−Π SP

gxk_Jgxk_ζξ

⎞ ⎟

⎠. 3.2

The following algorithm is an SQP-type algorithm for solving1.1, which is based on computing at each iteration a primal-dual stationary pointΔxk_{, λ}QP

k , μ QP

k of3.1. Algorithm 3.1

Step 1. Given an initial iterate point x1_{, λ}1_{, μ}1_{. Compute hx}1_{, gx}1_{, ∇fx}1_{, Jhx}1 _and

Jgx1_{. Setk:1.}

Step 2. If∇xLxk, λk, μk 0,hxk 0,gxk∈ SP, stop.

Step 3. Compute∇2_xxLxk_{, λ}k_{, μ}k_{, and find a solutionΔx}k_{, λ}k

QP, μkQPto3.1.

Step 4. Setxk1:xk _Δxk_{, λ}k1_:λk

QP, μk1 :μkQP.

Step 5. Computehxk1,gxk1, ∇fxk1,Jhxk1andJgxk1. Setk : k1 and go to step 2.

From item f of 7, Theorem 4.1 , we obtain the error between Δxk_{, λ}QP k , μ

QP k and x, λ, μdirectly.

Theorem 3.2. Suppose that f, h, gare twice continuously differentiable and their derivatives are

and SSOSC2.14hold atx. Then there exists a neighborhoodUofx, λ, μsuch that ifxk_{, λ}k_{, μ}k

in U, 3.1has a local solution Δxk _{together with corresponding Lagrangian multipliesλ}k

QP, μkQP

satisfying

_Δxk_λk

QP−λμkQP−μOxk, λ

k_{, μ}k_{−x, λ, μ. 3.3}

Now we are in a position to state that the sequence of primal-dual points generated by Algorithm 3.1has quadratic convergence rate.

Theorem 3.3. Suppose that f, h, gare twice continuously differentiable and their derivatives are

lo-cally Lipschitz in a neighborhood of a local solutionxto1.1. Suppose nondegeneracy condition2.12

and SSOSC 2.14 hold at x. Consider Algorithm 3.1, in which Δxk _{is a minimum norm}

station-ary point of the tangential quadratic problem 3.1. Then there exists a neighborhood U of x, λ, μ

such that, ifx1, λ1, μ1∈U,Algorithm 3.1is well defined and the sequence{xk, λk, μk}converges quadratically tox, λ, μ.

Proof. ByTheorem 3.2, we knowAlgorithm 3.1is well defined. Let

δk :xk, λk, μk−x, λ, μ, 3.4

then

Δxk Oδk

, λk1−λOδk

, μk1−μOδk

, 3.5

whereΔxk_{is the minimum norm solution to3.1, andλ}k1_λk

QP,μk1μkQPare the associated multipliers. Using Taylor expansion of3.2atx, λ, μ, noting that ∇xLx, λ, μ 0,xk1

xk _Δxk_{, and3.5, we obtain}

∇2

xxLx, λ, μ

xk1−xJhx∗λk1−λJgx∗μk1−μOδ2_k,

Jhxxk1−xOδ2_k. 3.6

As the projection operator Π_Sp

· is strongly semismooth, we have that there exists V ∈

∂Π_Sp

gx μsuch that

Π_Sp

gx μ Π_Sp

gxk JgxkΔxkμk_QP

Vgx μ−gxk− JgxkΔxk−μk_QP

Ogx μ−gxk− JgxkΔxk−μk_QP2.

3.7

Since

gx μ−gxk− JgxkΔxk−μk_QPJgxkx−xk1μ−μk_QPOδ2_k, 3.8

we have

Π_Sp gx

k _Jgxk_Δxk_μk QP Π_Sp

gx μ−VJgx

k_x−xk1 _μ−μk

QP Oδ2k.

Noting the fact thatgx Π_Sp

gx μ, by Taylor expansion of the third equation of3.2at

x, μ, we obtain

V −IJgxxk1−x Vμk1−μ Oδ2_k. 3.10

Therefore, we can conclude that

⎛ ⎜ ⎜ ⎝

∇2

xxLx, λ, μ Jhx∗ Jgx∗

Jhx 0 0

−Jgx VJgx 0 V ⎞ ⎟ ⎟ ⎠

⎛ ⎜ ⎜ ⎝

xk1−x λk1−λ μk1−μ

⎞ ⎟ ⎟

⎠Oδ2_k. 3.11

Since the nondegeneracy condition 2.12 and SSOSC 2.14 hold, we have from Propo-sition 2.4 that 3.11 implies the quadratic convergence of the sequence {xk_{, λ}k_,

μk_}.

3.2. The global convergence

The tangential quadratic problem constrained here is slightly more general than 3.1in the sense that the Hessian of the Lagrangian∇2xxLxk, λk, μkis replaced by some positive definite matrixMk. Thus the tangential quadratic problem inΔxnow becomes

minΔx∇fxkTΔx1 2Δx

T_M

kΔx, s.t. h

xkJhxkΔx0, gxkJgxkΔx∈ Sp.

3.12

The KKT systemof3.12is

∇fxk_MkΔxk_Jhxk∗_λk QPJg

xk∗_μk

QP0, h

xk_Jhxk_Δxk _0,

gxkJgxkΔxk−Π_Sp

gxkJgxkΔxkμk_QP0. 3.13

To obtain theglobal convergence, we use the Han penalty function given by 14 , as a merit function and Armijo line search. For problem1.1, the Han penalty function is defined by

Θσx fx σhx−σλmin

gx₋, 3.14

whereλmingxis the smallest eigenvalue ofgx,·₋denote min{·,0}andσ >0 is a positive constant.

The following proposition comes from2 directly.

Proposition 3.4. iIff,h,ghave a directional derivative atxin the directiond∈Rn_{, thenΘσ has}

also a directional derivative atxin the directiond. If, in addition,xis feasible for1.1, we have

Θσx;d fx;d σhx;d−σλmin

NTJgxdN, 3.15

whereN ν1, . . . , νr is the matrix whose columnsνiform an orthonormal basis ofKergx.

iiIfxis a feasible point of 1.1andΘσhas a local minimum atx, thenxis the local solution to1.1. Furthermore, iff,h,g are differentiable atxand nondegeneracy condition2.12holds atx, thenσ≥max{λ,tr−μ}.

To discuss the conditions ensuring the exactness of Θσ, we need the following lemma from3.10.

Lemma 3.5. Suppose nondegeneracy condition2.12and SSOSC2.14hold atx. Then there exists

c0 >0, such that for anyc > c0there exist a neighborhoodV ofxand a neighborhoodUofλ, μ, for

anyλ, μ∈U, the problem

minLcx, λ, μ s.t. x∈V 3.16

has a unique solution denotexcλ, μ. The functionxc·,·is locally Lipschitz continuous and semis-mooth onU. Furthermore, there existsρ >0, for anyλ, μ∈U,

_{x−xcλ, μ≤ρλ, μ−λ, μ/c, 3.17}

where

Lcx, λ, μ:fx hx, λc

2hx 2 1

2cΠSp

−μ−cgx2_F− μ2_F 3.18

is the augmented Lagrangian function with the penalty parametercfor1.1.

Theorem 3.6. Suppose thatf,h,gare twice differentiable around a local solutionxto1.1, at which nondegeneracy condition 2.12and SSOSC 2.14 hold. If σ > max{λ,tr−μ}, then Θσ has a strict local minimum atx.

Proof. For the definition of the projection operatorΠ_Sp

·, we have

Π_Sp

−μ−cgx−μ−cgx Π_Sp

cgx μ, 3.19

and for anyW ∈ Sp,c >0,

_ΠSp

cgx μ−cgx μ2_F ≤W −cgx μ2_F. 3.20

Then

_ΠSp

cgx μ−cgx2_F−2μ,Π_Sp

cgx μ−cgx

≤ −2μ, W−cgxW −cgx2_F 3.21

holds for anyW ∈ Sp. So takingμμandW cΠ_Sp

gx, we obtain that

_ΠSp

cgx μ−cgx2_F−2μ,Π_Sp

cgx μ−cgx

≤ −2cμ,Π_Sp

−gxc2Π_Sp

−gx2_F, 3.22

which implies

Lcx, λ, μ≤fx λ, hxc

2hx 2₋

μ,Π_Sp

−gx c

2ΠSp

−gx2_F

≤fx hx

λ c

2hx

λmax

Π_Sp

−gx

tr−μ c

2 p i1 λi

Π_Sp

−gx

.

Sinceσ >max{λ,tr−μ}, for any fixedc >0, there exists a neighborhoodVcofxsuch that

Lcx, λ, μ≤fx σhxσλmax

Π_Sp

−gx Θσx, ∀x∈Vc. 3.24

From Lemma 3.5, we know that there exist an r > c0 and a neighborhoodVr of x where x is a strict minimum of Lr·, λ, μ. So we can conclude that x is a strict minimum of Θσ on

Vc∩Vr.

Let us outline the line-search SQP-type algorithm that uses the merit functionΘσ· de-fined in3.14and the parameter updating scheme from14 , which is a generalized version to the algorithm in2 .

Algorithm 3.7

Step 1. Given a positive number σ > 0, ∈ 0,1/2, β ∈ 0,1/2. Choose an initial iterate

x1_{, λ}1_{, μ}1 _{∈ R}n_×Rl_{× S}p_{. Compute fx}1_,hx1_,gx1_{, ∇fx}1_,Jhx1_andJgx1_{. Set k :} 1, σ1σ.

Step 2. If∇xLxk, λk, μk 0, hxk 0, gxk∈ Sp, stop.

Step 3. Compute a symmetric matrixMkand find a solutionΔxk, λkQP, μkQPto3.12.

Step 4. Adaptσk.

ifσk−1≥max{tr−μk1,λk1}σ thenσk σk−1

elseσk max{1.5σk−1,max{tr−μk1,λk1}σ}

Step 5. Compute

wk :−

Δxk, MkΔxkμk_QP, gxkλk_QP, hxk−σkh

xkσkλmin

gxk₋.

3.25

Using backtracking line search rule to compute the step lengthαk:

Step 6. seti0,αk,0 1;

Step 7. if

Θσk

xkαΔxk≤Θσk

xkαwk 3.26

holds forααk,i, thenαk αand stop the line search.

Step 8. else, chooseαk,i1∈βαk,i,1−ββαk,i ;

Step 9. seti:i1, go tostep 7

Step 10. Setxk1:xkαkΔxk,λk1:λk_QP,μk1:μk_QP.

Step 11. Compute fxk1, hxk1, gxk1,∇fxk1,Jhxk1 and Jgxk1. Set k : k1 and go tostep 2.

Theorem 3.8. Suppose thatf,h,g are continuously differentiable and their derivatives are Lipschitz

continuous. ConsiderAlgorithm 3.7, if positive definite matricesMkandM−_k1are bounded, then one of

the following situations occurs:

ithe sequence{σk}is unbounded, in which case{λk1, μk1}is also unbounded;

iithere exists an indexk2such thatσk σfor anyk≥k2, and one of the following situations occurs:

a Θσxk→∞,

b∇xLxk, λk, μk→0, hxk→0, λmingxk₋→0,andμk1, gxk→0.

Acknowledgments

The research is supported by the National Natural Science Foundation of China under Project no. 10771026 and by the Scientific Research Foundation for the Returned Overseas Chinese Scholars, State Education Ministry of China.

References

1 B. Fares, D. Noll, and P. Apkarian, “Robust control via sequential semidefinite programming,”SIAM Journal on Control and Optimization, vol. 40, no. 6, pp. 1791–1820, 2002.

2 R. Correa and H. C. Ramirez, “A global algorithm for nonlinear semidefinite programming,”SIAM Journal on Optimization, vol. 15, no. 1, pp. 303–318, 2004.

3 M. Koˇcvara and M. Stingl, “Pennon: a code for convex nonlinear and semidefinite programming,” Optimization Methods & Software, vol. 18, no. 3, pp. 317–333, 2003.

4 M. Koˇcvara and M. Stingl, “Solving nonconvex SDP problems of structural optimization with stability control,”Optimization Methods & Software, vol. 19, no. 5, pp. 595–609, 2004.

5 D. Sun, J. Sun, and L. Zhang, “The rate of convergence of the augmented Lagrangian method for nonlinear semidefinite programming,”Mathematical Programming, 2008.

6 D. Noll, “Local convergence of an augmented Lagrangian method for matrix inequality constrained programming,”Optimization Methods and Software, vol. 22, no. 5, pp. 777–802, 2007.

7 D. Sun, “The strong second-order sufficient condition and constraint nondegeneracy in nonlinear semidefinite programming and their implications,”Mathematics of Operations Research, vol. 31, no. 4, pp. 761–776, 2006.

8 R. Mifflin, “Semismooth and semiconvex functions in constrained optimization,” SIAM Journal on Control and Optimization, vol. 15, no. 6, pp. 959–972, 1977.

9 L. Q. Qi and J. Sun, “A nonsmooth version of Newton’s method,”Mathematical Programming, vol. 58, no. 1–3, pp. 353–367, 1993.

10 E. H. Zarantonello, “Projections on convex sets in Hilbert space and spectral theory—I: projections on convex sets,” inContributions to Nonlinear Functional Analysis (Proceedings of a Symposium held at the Mathematics Research Center, University of Wisconsin, Madison, Wis., 1971), E. H. Zarantonello, Ed., pp. 237–341, Academic Press, New York, NY, USA, 1971.

11 E. H. Zarantonello, “Projections on convex sets in Hilbert space and spectral theory—II: spectral the-ory,” inContributions to Nonlinear Functional Analysis (Proceedings of a Symposium held at the Mathematics Research Center, University of Wisconsin, Madison, Wis., 1971), E. H. Zarantonello, Ed., pp. 343–424, Aca-demic Press, New York, NY, USA, 1971.

12 D. Sun and J. Sun, “Semismooth matrix-valued functions,”Mathematics of Operations Research, vol. 27, no. 1, pp. 150–169, 2002.

13 J. F. Bonnans and A. Shapiro,Perturbation Analysis of Optimization Problems, Springer Series in Opera-tions Research, Springer, New York, NY, USA, 2000.