R E S E A R C H
Open Access
New complexity analysis for primal-dual
interior-point methods for self-scaled
optimization problems
Bo Kyung Choi and Gue Myung Lee
**Correspondence:
[email protected] Department of Applied Mathematics, Pukyong National University, Busan, 608-737, Korea
Abstract
A linear optimization problem over a symmetric cone, defined in a Euclidean Jordan algebra and called a self-scaled optimization problem (SOP), is considered. We formulate an algorithm for a large-update primal-dual interior-point method (IPM) for the SOP by using a proximity function defined by a new kernel function, and we obtain the best known complexity results of the large-update IPM for the SOP by using the Euclidean Jordan algebra techniques.
MSC: 90C51; 90C25; 65K05
Keywords: Euclidean Jordan algebra; self-scaled optimization problem; primal-dual interior-point methods; kernel function; proximity function; complexity analysis; worst-case iteration bound
1 Introduction and preliminaries
Primal and dual interior-point methods (IPMs) have been well known as the most effec-tive methods for solving wide classes of optimization problems, for example, the linear optimization (LO) problem, the quadratic optimization problem (QOP), the semidefinite optimization (SDO) problem, the second-order cone optimization (SOCO) problem, and the convex optimization problem (CP).
The so-called barrier update parameter θ in algorithms for IPMs plays an important role in both theory and practice of IPMs. Usually, ifθis a constant independent of the di-mension of the problem, then the algorithm is called alarge-updatemethod. If it depends on the dimension, then the algorithm is said to be asmall-updatemethod. Large-update methods are much more efficient than small-update methods in practice [], but have a worst-case iteration bound. Such a gap between theory and practice has been referred to as irony of IPMs []. Recently, many authors have tried to reduce the gap of the worst-case iteration bound between the large-update IPM and the small-update IPM.
Using self-regular proximity functions instead of a classical logarithmic barrier func-tion, Penget al.[–] improved the complexity of large-update IPMs for the LO problem, the SDO problem, and the SOCO problem. Baiet al.[] introduced a new class of eligible kernel functions. The class was defined by some simple conditions on the kernel func-tion and its derivatives. The best iterafunc-tion bound for the LO problem, which was given by Baiet al.[], isO(√nlognlogn). Recently, Wanget al.[] obtained the complexity result
O(nlogn/) for the SDO problem based on a simple kernel function. Bai and Wang []
obtained the best known complexity result for the SOCO problem based on a parametric kernel function including the classical logarithmic function, the prototype regular kernel function, and the non-self-regular kernel function. Very recently, using the kernel func-tionφ(t) = (t– )/ + (et–q–– )/q, Choi and Lee [, ] have obtained the complexity
results of large-update primal-dual IPMs for SDO and SOCO,O(√n(logn)(q+)/qlogn/) andO(√N(logN)(q+)/qlogN/), respectively.
In this paper, we consider a linear optimization problem over a symmetric cone which is defined in a Euclidean Jordan algebra. Nesterov and Todd [] proposed first this kind of an optimization problem under the name of convex programming for self-scaled cones and established the polynomial complexity of the primal-dual interior point method using the so-called NT (Nesterov-Todd) direction []. We call the linear optimization problem over the symmetric cone the self-scaled optimization problem (SOP).
Faybusovich first studied the SOP in view of a Euclidean Jordan algebra and gave a the-oretical background for nondegeneracy assumptions and the uniqueness of solutions for Newton systems in IPMs for the SOP [], presented a short-step path-following algo-rithm for a quadratic programming problem defined on the intersection of a symmet-ric cone with an affine subspace [] and obtained complexity estimates for a long-step primal-dual interior-point algorithm for the optimization problem of the minimization of a linear function on a feasible set obtained as the intersection of an affine subspace and a symmetric cone []. SOPs include linear optimization problems, semidefinite optimiza-tion problems, second-order optimizaoptimiza-tion problems, and various combinaoptimiza-tions of these types of problems as special cases. Schmieta and Alizadeh [] extended primal-dual in-terior point algorithms for LOs, SDOs, and SOCOs to SOPs by using logarithmic barrier functions.
Baes raised an open question in his monograph [] as follows: The theory of self-regular functions has been created for linear programming by Jiming Peng, Cornelius Roos, and Tamás Terlaky []. They subsequently extended it to second-order programming and semidefinite programming separately using implicitly the aforementioned construction. However, the unified treatment of this theory using the Jordan algebraic framework is not accomplished yet.
Choi and Lee [] gave primal-dual interior point algorithms by using a very simple self-regular functionψ(t) =(t–t),t> for the SOP and gave partial answers for the ques-tion of Baes. Very recently, Vieira [, ] gave complete answers for the open quesques-tion of Baes by proving thee-convexity property of eligible kernel functions and, in particular, he presented the iteration complexity results for ten eligible kernel functions. Among ten kernel functions in [], the best iteration complexity for a large-update method was ob-tained forψ(t) =t–+t–q–q–withq=logr, and its iteration complexity isO(√rlogrlogr), which is the best known one.
In this paper, we define a new eligible kernel functionψ(t) =t–+ep(t –q–)
–
pq ,p and
But we use Proposition . in [] obtained from the technique of Sun and Sun [] instead of using Proposition . in [].
This paper is organized as follows. In Section , we introduce our kernel functions, for-mulate the Newton system for the SOP, and present a useful inequality for our proximity function. In Section , we give an algorithm for the SOP and calculate an upper bound for the proximity function afterμ-update. We calculate an upper bound for difference between proximity functions after one step in inner iterations and then determine our default step size for search directions. We present a worst-case iteration bound for our large-update primal-dual interior point method for the SOP.
Now, we give definitions and preliminary properties for a Euclidean Jordan algebra which are found in [] and will be used in the next sections.
Definition . ([]) A finite-dimensional real vector spaceV is called an algebra if a bilinear mapping (x,y)→x◦yfromV×VtoV is defined.
An algebraVis called a Jordan algebra if the following hold:
(i) commutativity: for allx,y∈V,x◦y=y◦x;
(ii) Jordan’s axiom: for allx,y∈V,x◦(x◦y) =x◦(x◦y), wherex=x◦x.
A Jordan algebraVis said to be Euclidean if
(iii) x+y= ⇒x=y= , equivalently, there exists an inner product(·|·)onV such that(x◦y|z) = (y|x◦z).
A Jordan algebraV is simple if it does not contain any non-trivial ideal. The Jordan algebra may not be associative, but it is power-associative,i.e.,xp◦xq=xp+q. We assume a Jordan algebraV has an identity element,i.e., there existsesuch thatx◦e=e◦x=x. SinceV is finite-dimensional, givenx∈V, there exists a minimal positive integerksuch that the vectorse,x, . . . ,xkare linearly dependent. Denote this integerm(x). We define the
rank ofVas
rank(V) =r=maxm(x)|x∈V.
An element x∈V is said to be invertible if there exists an elementy∈R[x] such that x◦y=e, whereR[x] is the algebra overRof polynomials in one variable with coefficients inR. It is defined byx–. An elementv∈V is called idempotent ifv=v. For an element x∈V, letL(x) be a linear map ofVdefined asL(x)y=x◦y. The cone of squares
:=x|x∈V
is a symmetric cone; the following conditions hold:
(i) for every pair ofx,y∈int, there is an invertible linear transformationL:V→V
such thatL() =andL(x) =y;
(ii) *=, where*:={y∈V|(x,y),for anyx∈}.
Let=int. Then={x|x∈Vis invertible}={x∈V|L(x) is positive definite}.
Definition .([]) Letc, . . . ,ck∈V. Then{c, . . . ,ck}is said to be a Jordan frame ifci,
following properties hold:
⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩
c i =ci,
ci◦cj= ifi=j,
k i=ci=e.
Theorem . (Theorem III.. in []) For every x∈ V, there exist a Jordan frame
{c(x), . . . ,cr(x)}and real numbersλ(x), . . . ,λr(x)such that
x=λ(x)c(x) +· · ·+λr(x)cr(x). ()
The numbersλi(x), for alli= , . . . ,r, are said to be the eigenvalues ofx, and () is called
the eigenvalue (or spectral) decomposition ofx. Now, it is possible to extend the defini-tion of any real-valued funcdefini-tionψ(·) to elements of the Euclidean Jordan algebra via their eigenvalues:
ψ(x) :=ψ λ(x)
c(x) +· · ·+ψ λr(x)
cr(x). ()
Particularly, we have some examples as follows:
(i) Square root:x/=λ/
(x)c(x) +· · ·+λ/r (x)cr(x)if allλi(x). (ii) Inverse:x–=λ–
(x)c(x) +· · ·+λ–r (x)cr(x)if allλi(x)= . (iii) Square:x=λ
(x)c(x) +· · ·+λr(x)cr(x).
From the above examples, we know that forx∈,λi(x/) =λi/(x) and forx∈,λi(x–) = λ–i (x). Let us denote byψ(x) the derivative ofψ(x) with respect toλi(x):
ψ(x) :=ψ λ(x)
c(x) +· · ·+ψ λr(x)
cr(x). ()
In the Jordan algebra, we define the determinant ofxand the trace ofxas follows:
det(x) =
r
i=
λi(x), tr(x) = r
i= λi(x).
SinceVis a Euclidean Jordan algebra, x,y:=tr(x◦y) is a scalar product onV(see Propo-sition III.. in []). The following lemma is called the second Pierce decompoPropo-sition the-orem which will be used in Section .
Lemma .(Theorem IV.. in [], Theorem .. (Second Pierce decomposition
theo-rem) in []) Let{c, . . . ,cr}be a Jordan frame of V.If
Vij:=
⎧ ⎨ ⎩
{vij|ci◦vij=vij} if i=j, {vij|ci◦vij=vij} ∩ {vij|cj◦vij=vij} if i=j,
we have
(i) V=ijrVij;
(ii) Vij◦Vkl={},if{i,j} ∩ {k,l}=∅; (iii) Vij◦Vjk⊂Vik,ifi=k;
Consider the following self-scaled optimization problem (SOP):
(P) Minimize c,x
subject to ai,x=bi, i= , . . . ,m,
x∈,
and its dual problem:
(D) Maximize
m
i=
biyi
subject to
m
i=
yiai+s=c,
s∈, y∈Rm,
wherec,a, . . . ,am∈V andb∈Rm are given. We callx∈primal feasible if ai,x=bi
fori= , . . . ,m. Similarly, (y,s)∈Rm× is called dual feasible ifm
i=yiai+s=c. Let
Ax= (a,x, . . . , am,x)T for any x∈V. Then A:V →Rm is a linear transformation.
Throughout this paper, we assume thatAis surjective. Then its adjointAT is injective andATy=m
i=yiai, wherey= (y, . . . ,ym)T∈Rm. So, we can reformulate (P) and (D) as
follows:
(P) Minimize c,x
subject to Ax=b, x∈,
and its dual problem:
(D) Maximize bTy
subject to ATy+s=c,
s∈, y∈Rm.
We can check that weak duality between (P) and (D) holds, that is,inf(P)sup(D). From now on, we assume that both (P) and (D) satisfy the interior-point condition (IPC), that is, there exists (x,y,s) such thatAx=b,x∈,ATy+s=c,s∈. Then there exists
a pair of optimal solutions (x,y,s) of (P) and (D), andinf(P) =sup(D) [, ]. The following lemma is well known [, , , ].
Lemma . For x,s∈V,the following statements are equivalent:
(i) x,s∈and x,s= ; (ii) x,s∈andx◦s= .
Using Lemma ., we can check (see Proposition . in []) that finding a pair of optimal solutions (x,y,s) of (P) and (D) is equivalent to solving the following Newton system:
⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩
Ax=b,
ATy+s=c,
x◦s= ,
x,s∈, y∈Rm.
The basic idea of primal-dual IPMs is to replace the third equation in (), the so-called complementarity conditionfor the SOP, by the parameterized system with a positive pa-rameterμ:
⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩
Ax=b,
ATy+s=c,
x◦s=μe,
x,s∈, y∈Rm.
()
For eachx∈V, we define the quadratic representation as follows:
Qx:= L(x) –L x
.
Lemma .([]) Let x,s∈and p be invertible.Then x◦s=μe if and only if Qpx◦Qp–s=
μe.
Proposition .(Proposition in []) If x,s∈,then Qxs∈.
Letx,s∈. Then there uniquely existsp∈such thatQpx=s[, ]. So, we can choosep∈such thatQpx=Qp–s. Such a choice exists and is unique, and leads to the Nesterov-Todd (NT) method.
From Lemma ., the system () becomes
⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩
Ax=b,
ATy+s=c,
Qpx◦Qp–s=μe,
x,s∈, y∈Rm.
()
Then, for eachμ> , the parameterized system () has a unique solution (x(μ),y(μ),s(μ)) [, ], which is called a μ-center of (P) and (D). The set ofμ-centers, that is, C=
{(x(μ),y(μ),s(μ))|μ> }, is said to be thecentral pathof (P) and (D). Therefore, asμtends to zero, (x(μ),y(μ),s(μ)) converges to a pair of optimal solutions of (P) and (D) [, ].
In general, IPMs for the SOP consist of two strategies. The first one, which is called the inner iteration scheme, is to keep the iterative sequence in a certain neighborhood of the central path or to keep the iterative sequence in a certain neighborhood of theμ-center. And the second one, called the outer iteration scheme, is to decrease the parameterμto
μ+:= ( –θ)μfor someθ∈(, ).
2 Proximity functions and search directions
Newton’s method is a well-known procedure to solve a system of nonlinear equations. Most IPMs for solving the SOP employ different search directions together with suitable strategies for following the central path appropriately.
Assume that a starting point (x,s) in a certain neighborhood of the central path
and linearize the Newton system for () by replacingx,y,swithx+:=x+x,y+:=y+y,
s+:=s+s, respectively. Then we get the following system in []:
⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩
Ax= ,
ATy+s= ,
Qpx◦Qp–s+Qpx◦Qp–s=μ+e–Qpx◦Qp–s.
()
To describe our new search direction, we need more notations:
¯
A:=√
μAQp–, v:=
√μQpx=
√μQp–s,
dx:=√
μQpx, ds:=
√μQp–s.
()
In this case,
p=Qx/(Qx/s)–/
–/
=Qs–/(Qs/x)/
–/
. ()
From Proposition .,v∈. Hence,L(v) is positive definite. Thus, the system () is equiv-alent to the following system:
⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩
¯
A dx= ,
¯
ATy+ds= ,
dx+ds=v––v.
()
We say that the above (dx,y,ds) is called the NT search direction for the SOP. Further-more, dx,ds= , which is coming from the first and second equations of () or from the orthogonality ofxands.
For our IPM, we use the following new eligible kernel function:
ψ(t) =t
–
+
ep(t–q–)–
pq , p andq fort> . ()
Please see the definition of an eligible function in []. The new kernel function () sat-isfies
ψ(t) > , ψ(t) < and lim
t→+ψ(t) =tlim→∞ψ(t) =∞.
Note thatψ() =ψ() = . Thenψ(t) is determined:
ψ(t) =
t
ξ
ψ(ζ)dζdξ. ()
The proximity function (measure) for (P) and (D) is
(x,s;μ) :=(v) :=tr ψ(v)=
r
i=
ψ λi(v)
whereψ(v) is defined by (). Note that(v) = , ifv=e(i.e.,x◦s=μe) and(v) > , otherwise. Replacing the right-hand side of the last equation in () by –ψ(v), we have the following system from ():
⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩
¯
A dx= ,
¯
ATy+ds= ,
dx+ds= –ψ(v).
()
LetX={x∈V| ¯Ax= }. ThenX⊥={ ¯ATy|y∈Rm}. Hence, the system () has a unique
solution.We introduce the norm-based proximity measure as follows:
σ:=dx+ds=ψ(v)=dx+ds. ()
The following lemma gives a lower bound ofσin terms of(v).
Lemma . For any v∈,
σ(v).
Proof Since () satisfies ψ(t)(ψ(t))andσ=r
i=(ψ(λi(v))),
(v)σ.
This completes the proof.
Also, our new kernel function () satisfies the following exponential convexity property.
Lemma . Let t> and t> .Then
ψ(√tt)
ψ(t) +ψ(t)
.
The following proposition can be found in [], but for the completeness, we give its proof.
Proposition .(Theorem . in []) Let be the proximity function defined in(), then for any x,s∈,
(Qx/s)/
(x) +(s)
.
Proof SinceQx/s∈,
λi (Qx/s)/
=λ/i (Qx/s) and (Qx/s)/
=
r
i=
ψ λ/i (Qx/s)
.
By Theorem . in [],
k
i=
λi(Qx/s)
k
i=
and
r
i=
λi(Qx/s) =
r
i=
λi(x)λi(s).
Thus,
k
i=
λ/i (Qx/s)
k
i=
λ/i (x)λi/(s), fork= , . . . ,r– ,
and
r
i=
λ/i (Qx/s) =
r
i=
λ/i (x)λ/i (s).
Letαi=λ/i (Qx/s) andβi=λ/i (x)λ/i (s). Thenαi> andβi> . Moreover, since these
conditions satisfy the assumptions of Corollary .. in [] and (iii) in Corollary .. in [] with our kernel function (),
r
i=
ψ λ/i (Qx/s)
r
i=
ψ λ/i (x)λ/i (s).
By Lemma ., we obtain the following result:
r
i=
ψ λ/i (x)λ/i (s)
r
i=
ψ λi(x)
+
r
i=
ψ λi(s)
=
(x) +(s)
.
3 Algorithm and its complexity analysis
Now, we explain our algorithm for the large-update primal-dual IPM for the SOP. Assum-ing that a startAssum-ing point in a certain neighborhood of the central path is available, we can set out from this point. Then, we will go to the outer ‘while loop’. Ifμsatisfiesrμ, then it is reduced by the factor –θ, whereθ∈(, ). Then, we make use of the inner ‘while loop’, and we repeat the procedure until we find iterates that are ‘close’ to (x(μ),y(μ),s(μ)), that is, the proximity(x,s;μ) <τ. Here, we apply Newton’s method targeting at the new
μ-centers to decide a search direction (x,y,s). We return to the outer ‘while loop’. The whole process is repeated untilμis small enough, say untilrμ<.
The choice of the step sizeαis another crucial issue in the analysis of the algorithm. It has to be taken so that the closeness of the iterates to the currentμ-center can improve by a sufficient amount. In the algorithm, the inner ‘while loop’ is called theinner iteration and the outer ‘while loop’ is called theouter iteration. Each outer iteration consists of an update of the parameterμand a sequence of (one or more) inner iterations. The total number of inner iterations is the worst-case iteration bound for our algorithm.
The algorithm for our large-update primal-dual IPM for the SOP is given as follows: Primal-dual algorithm for the SOP
Inputs
a variable damping factorα;
a fixed barrier update parameterθ∈(, ); (x,s) andμ= such that(x,s;μ)τ.
begin
x:=x;s:=s;μ:=μ;
whilerμdo
begin
μ:= ( –θ)μ;
while(x,s;μ)τ do begin
Solve the system () forx,y,s; Determine a step sizeα;
x:=x+αx; y:=y+αy; s:=s+αs; end
end end
3.1 Bound of the proximity function after
μ
-updateWe have(v)τ before the update ofμwith the factor –θ at the start of each outer iteration. After updatingμin an outer iteration, the vectorvis divided by the factor√ –θ, which in general leads to an increase in the value of(v). Then during the inner iteration, the value of(v) decreases until it passes the thresholdτ.
As we mentioned, our kernel function () is eligible. To obtain an upper bound for a
μ-updated proximity function in each outer iteration in the algorithm, we use the well-known Lemma ., which can be induced from the decreasing part of the kernel function, instead of using theorems which can be obtained from some properties for eligible func-tions (for example, Theorem . in [] and Theorem . in []). Both of the following lemmas make our analysis in the outer while loop easy. And we will show a theorem that an upper bound for(√
–θv) is expressed with(v) by using the following two lemmas.
Lemma . Letβ.Then
ψ(βt)ψ(t) +(β
– )
t
.
Proof Defineψb(t) := e
p(t–q–)–
pq . Thenψb(t) is monotonically decreasing int. So, we can
easily obtain
ψ(βt) = β
t–
+ψb(βt) = t–
+ψb(t) +
βt–t
+ψb(βt) –ψb(t)
ψ(t) +(β
– )
t
.
Lemma . For any v∈,then
Proof Since ep(t –q–)
pq is positive andpq, the kernel function () has a lower bound as
follows:
ψ(t)t
–
–
pq
t
– – .
This implies ri=λi(v)(v) + r.
Theorem . Letθbe such that <θ< .Then,for any v∈,
√
–θv
–θ (v) +r
.
Proof From Lemma . withβ=√
–θ and Lemma .,
√
–θv
=
r
i= ψ
√
–θλi(v)
(v) +
–θ –
v
(v) + θ
–θ (v) + r
–θ (v) +r
,
the last inequality comes fromθ∈(, ).
By the assumption(v)τjust before the update ofμ,
√
–θv
–θ(τ+r).
We define
L(r,θ,τ) =
–θ(τ+r).
Sinceτ=O(r) andθ=(),
L=O(r).
3.2 Determining a default step size
In this section, we compute the feasible step sizeα such that the proximity function is decreasing and is bound for the decrease during inner iterations; then we give our default step sizeα¯;α¯ = (( + σ( +pq+q)( +p–logσ)(q+)/q))–. We will show that the step
size not only keeps the iterates feasible but also gives rise to a sufficiently large decrease in the barrier function(v) in each inner iteration. Let us denote the difference between the proximity before and after one step by a function of the step size, that is,
f(α) :=(v+) –(v). ()
The main task in the rest of this section is to study the decreasing behavior off(α). Now, in equation (),v+andp+are determined byx,sin () and () replaced byx+:=
x+αx,s+:=s+αs, respectively, which is as follows:
v+:=
√μQp+(x+αx) =
√μQp–
Lemma .(Proposition II.. in []) Let x and s be elements in V.Then
(i) (Qxs)–=Qx–s–ifxandsare invertible.
(ii) QQsx=QsQxQs.
Lemma .([]) Let x,s,p∈.Then
(i) Qx/sandQs/xhave the same eigenvalues.
(ii) Qx/sandQ(Qpx)/(Qp–s)have the same eigenvalues.
The following proposition was given by Vieira in [] (see Proposition . in []), but we provide its proof using Lemma . and Lemma ..
Proposition . Letbe the proximity function defined in().Then we have
(v+) =
Q(v+αdx)/(v+αds)
/
.
Proof FromQp–s=Qpxand (i) in Lemma ., we know thatQpxandQx/p have the same eigenvalues. By the definition ofpand (ii) in Lemma .,
Qx/p=Qx/ Q/x (Qx/s)–/
–
=Qx/Qx–/(Qx/s)/= (Qx/s)/.
Then we can find Qp–
+ s+ and (Qx/+ s+)
/ have the same eigenvalues. Here, √μv +=
Qp–
+s+. We know thatx+=
√μ
Qp–(v+αdx) ands+=√μQp(v+αds), by the definition () and by (ii) in Lemma ., then (Qx/
+ s+)
/=√μ(Q(Q
p–(v+αdx))/(Qp(v+αds)))/and
√
μ(Q(v+αdx)/(v+αds))/have the same eigenvalues. Therefore, the proximity function
satisfies the equality.
Then Proposition . and Proposition . imply the following inequality:
(v+)
(v+αdx) +
(v+αds).
So, we can definef(α):
f(α)f(α) :=
(v+αdx) +(v+αds)
–(v).
To facilitate the forthcoming analysis, we also define, for anyx∈V,
λmin(x) :=min
λi(x)|i= , . . . ,r
.
The following lemma is obtained from Lemma in [] so that we can get the common lower bound of eigenvalues ofv+αdxandv+αds, whereα satisfiesv+αdx∈and v+αds∈.
Lemma . For anyα∈(,λmin(v)
σ ),
λmin(v+αdx)λmin(v) –ασ and λmin(v+αds)λmin(v) –ασ,
Proof Letαbe a fixed number in (,λmin(v)
σ ). From Lemma in [],
λmin(v+αdx)λmin(v) –αdx.
Sinceσdx, we have
λmin(v+αdx)λmin(v) –ασ.
Similarly, we obtain
λmin(v+αds)λmin(v) –ασ.
The proof of the following proposition can be found in [], but for the completeness, we give its detailed proof.
Proposition .([]) Suppose that the functionsψ(x)and(x)are defined by()and (),respectively.Then,for anyα∈(,λmin(v)
σ ),
d dαf(α) =
tr ψ
(v+αdx)◦dx+ tr ψ
(v+αds)◦ds,
d
dαf(α)
max
ψ λi(v+αdx),λj(v+αdx)
|i,j= , . . . ,rdx
+
max
ψ λi(v+αds),λj(v+αds)
|i,j= , . . . ,rds,
where
ψ λi(·),λj(·)
=
⎧ ⎨ ⎩
ψ(λi(·)) ifλi(·) =λj(·),
ψ(λi(·))–ψ(λj(·))
λi(·)–λj(·) ifλi(·)=λj(·).
Proof Using Lemma . in [], we have
d
dαψ(v+αdx)
=
r
i=
ψ λi(v+αdx),λi(v+αdx)
ci(v+αdx),dx
ci(v+αdx)
+
j<lr
ψ λj(v+αdx),λl(v+αdx)
×cj(v+αdx)◦ cl(v+αdx)◦dx
. ()
Then we have
d
dαtr ψ(v+αdx)
= d
dα
ψ(v+αdx),e
=
d
dαψ(v+αdx),e
=tr
d
dαψ(v+αdx)
=tr
r
i=
ψ λi(v+αdx)
ci(v+αdx),dx
ci(v+αdx)
(by associativity of trace)
=
r
i=
ψ λi(v+αdx)
ci(v+αdx),dx
trci(v+αdx)
=
r
i=
ψ λi(v+αdx)
ci(v+αdx),dx
tr ci(v+αdx)
.
Then from Baes [, ] we know thattr(ci(v+αdx)) = , and hence, from the definition
(), we get
d
dαtr ψ(v+αdx)
=tr ψ(v+αdx)◦dx.
Thus, we have
d dαf(α) =
tr ψ
(v+αdx)◦dx+ tr ψ
(v+αds)◦ds.
So, the first equality holds.
For the second inequality, we will use () by replacingψ byψ.
d
dαtr ψ(v+αdx)
= d
dαtr ψ
(v+αdx)◦dx
=tr
d dαψ
(v+αdx)◦dx
=tr
r
i=
ψ λi(v+αdx),λi(v+αdx)
ci(v+αdx),dx
ci(v+αdx)
+
j<lr
ψ λj(v+αdx),λl(v+αdx)
×cj(v+αdx)◦ cl(v+αdx)◦dx
◦dx
.
Here, letdx=rj=λj(dx)cj(dx). Then we have
r
i=
tr ci(v+αdx)◦dx
=
r
i=
tr
r
j=
λj(dx)ci(v+αdx)◦cj(dx)
=
r
i=
r
j=
λj(dx)tr ci(v+αdx)◦cj(dx)
Sinceci(v+αdx) andcj(dx) are inwhich is a self-dual cone, then
tr ci(v+αdx)◦cj(dx)
, fori,j= , . . . ,r.
Furthermore,rj=tr(ci(v+αdx)◦cj(dx)) =tr(ci(v+αdx)) = . Then we have
r
i=
tr ci(v+αdx)◦dx
= r i= r j=
tr ci(v+αdx)◦cj(dx)
λj(dx)
r i= r j=
λj(dx)tr ci(v+αdx)◦cj(dx)
= r j=
λj(dx)
r
i=
tr ci(v+αdx)◦cj(dx)
.
Sinceri=tr(ci(v+αdx)◦cj(dx)) =tr(cj(dx)) = , we have
r
i=
tr ci(v+αdx)◦dx
dx. ()
Now, we decomposedx along Lemma . such asdx=ikrdxik for the system of
idempotent{c(v+αdx), . . . ,cr(v+αdx)}. Then, forj<l,
tr cj(v+αdx)◦dx
◦ cl(v+αdx)◦dx
=tr
cj(v+αdx)◦
ikr
dxik
◦
cl(v+αdx)◦
ikr
dxik
=tr
cj(v+αdx)◦ r i= dxji ◦
cl(v+αdx)◦ r
i=
dxli
by (ii) in Lemma .
=tr
dxjl◦dxlj
by (iii) and (iv) in Lemma .
= tr dx
jl
.
This means, for eachj<l,
tr cj(v+αdx)◦dx
◦ cl(v+αdx)◦dx
.
Moreover, we have,
j<lr
tr cj(v+αdx)◦dx
◦ cl(v+αdx)◦dx
j<lr
tr cj(v+αdx)◦dx
◦ cl(v+αdx)◦dx
+ r i=
tr ci(v+αdx)◦dx
= tr
r
i=
ci(v+αdx)◦dx
= tr dx
= dx
. ()
Since for eachi, (tr(ci(v+αdx)◦dx))are nonnegative and for eachj,lwithj<l,tr((cj(v+ αdx)◦dx)◦(cl(v+αdx)◦dx)) are nonnegative, we get from () and ()
d
dαtr ψ(v+αdx)
maxψ λi(v+αdx),λj(v+αdx)
|i,j= , . . . ,rdx.
Similarly,
d
dαtr ψ(v+αds)
maxψ λi(v+αds),λj(v+αds)
|i,j= , . . . ,rds.
From the definition off(α),
d
dαf(α) =
d
dα
tr ψ(v+αdx)
+
tr ψ(v+αds)
.
Thus, we have the conclusion.
The next result presents an upper bound for the second derivative off(α) which is usable
for establishing the polynomial complexity of the algorithm.
Proposition . For anyα∈(,λmin(v)
σ ),
f(α) ψ
λ
min(v) –ασ
σ.
Proof Sinceψ(t) is a decreasing function ont∈(,∞), using Lemma . and the mean value theorem, we have
ψ λmin(v) –ασ
maxψ λi(v+αdx),λj(v+αdx)
|i,j= , . . . ,r
and
ψ λmin(v) –ασ
maxψ λi(v+αds),λj(v+αds)
|i,j= , . . . ,r.
Thus, by Proposition .,
d
dαf(α)
ψ
λ
min(v) –ασ
dx+ ψ
λ
min(v) –ασ
ds
=
ψ
λ
min(v) –ασ
We can easily check that f() = andf() = –
σ
. By Proposition ., we obtain an
upper boundf(α) forf(α) as follows:
f(α) =f() +f()α+
α
ξ
f(ζ)dζdξ
f(α) :=f() +f()α+
σ
α
ξ
ψ λmin(v) –ασ
dζdξ.
Note thatf() = . Furthermore, sincef(α) = –σ
+ σ
(ψ(λmin(v)) –ψ(λmin(v) –ασ)), we
havef() = –σ
which is the same value off(), andf(α) =σ
ψ(λmin(v) –ασ) which
is increasing onα∈[,λmin(v)
σ ). Usingf() =f() andf(α)f(α), we can easily check
that
f(α) =f() +
α
f(ξ)dξf(α).
This relation gives that
f(α), iff(α).
To compute the feasible step sizeαsuch that the proximity measure is decreasing when we take a new iterate for fixedμ, we want to calculate the step sizeαwhich satisfies that f(α) holds withαas large as possible. Sincef(α) > , that is,f(α) is monotonically increasing atα, the largest possible value atαsatisfyingf(α) occurs whenf(α) = , that is,
–ψ λmin(v) –ασ
+ψ λmin(v)
=σ
. ()
Sinceψ(t) is monotonically decreasing, the derivative of the left-hand side in () with respect toλmin(v) is
–ψ λmin(v) –ασ
+ψ λmin(v)
< .
So, the left-hand side in () is decreasing atλmin(v). This implies that ifλmin(v) becomes
smaller, thenαgets smaller with fixedσ. Note that
σ=
n
i=
ψ λi(v)
ψ λmin(v)–ψ λmin(v)
and the equality is true if and only ifλmin(v) is the only coordinate in (λ(v), . . . ,λr(v)) which
is different from andλmin(v) < , that is,ψ(λmin(v)) < . Hence, the worse situation for
the largest step size occurs whenλmin(v) satisfies
–ψ λmin(v)
=σ. ()
From now on, we denote thatρ: [,∞)→(, ] is the inverse function of the restriction of –ψ(t) in the interval (, ]. Then () implies
λmin(v) =ρ(σ). ()
By using () and (), we immediately obtain
–ψ λmin(v) –ασ
=
σ.
By the definition ofρand (), the largest step sizeαof the worse case is given as follows:
α*=ρ(σ) –ρ(
σ)
σ . ()
For the purpose of finding an upper bound off(α), we need a default step sizeα¯that is the lower bound of theα*and consists ofσ.
Lemma . Letσ.Then,for <tρ(σ),
ψ(t) + σ( +pq+q)
+ plogσ
q+
q .
Proof Fromψ(t) =t–t–q–·ep(t–q–), let –ψ
b(t) =t–q–·ep(t
–q–)
and letρ: [,∞)→(, ] denote the inverse function of the restriction of –ψb(t) to the interval (, ]. Letρ(
σ) =˜t.
Then <t˜ and
σ= –ψ(t˜) = –t˜–ψb(˜t). So, –ψb(˜t) =t˜+σ + σσ. Sinceρis
a decreasing function, (ρ(σ) =˜t=)ρ(–ψb(˜t))ρ(σ). Letρ(σ) =ˆt. Then
σ= –ψb(ˆt) = ρ(σ)–q–·ep((ρ(σ))–q–) ()
implies
ep((ρ(σ))–q–)= σ ρ(σ)q+σ ⇒ p ρ(σ)–q– logσ
⇒ ρ(σ)
+ plogσ
–q
, ()
ψ(ˆt) = + (q+ )ˆtq+pqˆt–q–·ep(ˆt–q–)· ˆt–q–= + (q+ )ˆtq+pq –ψb(ˆt)· ˆt–q–
= + (q+ )ˆtq+pq·σ· ρ(σ)–q– + σ( +pq+q)
+ plogσ
q+
q ,
the last inequality comes fromˆt∈(, ] and ().
Now, we present a lower bound of the value ofα*.
Theorem . Letα*be as defined in().Then
α*
Proof Since –ψ(ρ(σ)) =σ, taking the derivative ofσat both sides, we get
ρ(σ) = –
ψ(ρ(σ)).
Moreover, we have
α*=
σ
σ
σ
ρ(ξ)dξ=
σ
σ
σ
ψ(ρ(ξ))dξ
σ ξ
ψ(ρ(σ))
! σ
σ
=
ψ(ρ(σ)),
where the inequality follows fromσξσandρandψare monotonically decreasing.
Also, by Lemma ., we can complete the proof.
For usingα¯ as the default step size in the algorithm for the SOP, define theα¯as follows:
¯
α=
( + σ( +pq+q)( +plogσ)q+q )
. ()
We will useα¯as the default step size in our algorithm.
3.3 Decrease of the proximity function during an inner iteration
Now, we show that our proximity functionwith our default step sizeα¯is decreasing. It can be easily established by using the following result.
Lemma .([]) Let h(t)be a twice differentiable convex function with h() = ,h() < and let h(t)attain its(global)minimum at t*> .If h(t)is increasing for t∈[,t*],then
h(t)th ()
, tt
*.
Sincef(α) satisfies assumptions of the above lemma,
f(α)f(α)f(α)
f()
α for all αα
*.
Sincef() = –σ, we can obtain the upper bound for the decreasing value of the proximity in the inner iteration by Lemma ..
Theorem . Letα¯ be the default step size as defined in().Then we have
f(α¯)– ·
√
+ √( +pq+q)( +plog√)q+q .
Proof Sincef() = –σ
andα¯∈[,α*], we have
f(α¯) α¯f
() =
·
( + σ( +pq+q)( +plogσ)q +
q )
·
–σ
= – ·
σ
+ σ( +pq+q)( +plogσ) q+
This expresses the decrease in one inner iteration in terms ofσ. Since the decrease de-pends monotonically onσ, we can express the decrease in terms of=(v) by Lemma . as follows:
f(α¯)– ·
+ √( +pq+q)( +plog√) q+
q
– ·
√ ·√ √
+ √( +pq+q)( + plog
√
)
q+
q
= – ·
√
+ √( +pq+q)( +plog√)
q+
q ,
where the inequality follows fromτ. The theorem is satisfied.
3.4 Iteration bound
We need to count how many inner iterations are required to return to the situation where
(v)τ after aμ-update. We denote the value of(v) afterμ-update as; the
subse-quent values in the same outer iteration are denoted ask,k= , . . . . IfKdenotes the total
number of inner iterations in the outer iteration, then we have
L(r,θ,τ) =O(r), K–>τ, Kτ
and according to Theorem .,
k+k–
+ √( +pq+q)( +plog√)
q+
q
k.
At this stage, we invoke Lemma in [].
Lemma .([]) Let t,t, . . . ,tKbe a sequence of positive numbers such that
tk+tk–βt–
γ
k , k= , , . . . ,K– ,
whereβ> and <γ .Then
K t
γ
βγ.
Lettingtk=k, β=
+√(+pq+q)(+plog√)
q+
q
andγ =, we can get the following
lemma from Lemma ..
Lemma . Let K be the total number of inner iterations in the outer iteration.Then we have
K
+ √( +pq+q)
+ plog
√
q+
q
/,
Now, we estimate the total number of iterations of our algorithm.
Theorem . Ifτand <θ< ,the total number of iterations is not more than
"
+ √( +pq+q)
+ plog
√
q+
q
/
#"
θlog
r
#
.
Proof In the algorithm,rμ,μk:= ( –θ)kμandμ= . By simple computation, we
have
k
θ log
r
.
Therefore, the number of outer iterations is bounded above by
θ log
r
.
Multiplication of this result by the number in the above lemma satisfies the theorem.
Since/=O(√r), if we takep=O(logr) andq= , then we can get the best known upper bound for the total number of inner iterations in the outer iteration is
O(√rlogr).
Also, we take forθ a constant (not depending on r), namely θ =(). Withτ =O(r), the best complexity of the primal-dual interior-point method for a linear optimization problem based on our new proximity function withp=lograndq= is given by
O
√
rlogrlogr
.
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
The authors, together discussed and solved the problems in the manuscript. All authors read and approved the final manuscript.
Acknowledgements
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MEST) (No. 2012-0006236).
Received: 30 June 2012 Accepted: 7 November 2012 Published: 26 November 2012 References
1. Andersen, ED, Gondzio, J, Mészáros, C, Xu, X: Implementation of interior point methods for large scale linear programming. In: Terlaky, T (eds.) Interior Point Methods of Mathematical Programming, pp. 189-252. Kluwer Academic, Dordrecht (1996)
2. Renegar, J: A Mathematical View of Interior-Point Methods in Convex Optimization. MPS/SIAM Ser. Optim. SIAM, Philadelphia (2001)
3. Peng, J, Roos, C, Terlaky, T: Primal-dual interior-point methods for second-order conic optimization based on self-regular proximities. SIAM J. Optim.13, 179-203 (2002)
5. Peng, J, Roos, C, Terlaky, T: Self-Regularity: A New Paradigm for Primal-Dual Interior-Point Algorithms. Princeton University Press, Princeton (2002)
6. Bai, YQ, Ghami, ME, Roos, C: A comparative study of kernel functions for primal-dual interior-point algorithms in linear optimization. SIAM J. Optim.15, 101-128 (2004)
7. Wang, GQ, Bai, YQ, Roos, C: Primal-dual interior-point algorithms for semidefinite optimization based on a simple kernel function. J. Math. Model. Algorithms4, 409-433 (2005)
8. Bai, YQ, Wang, GQ: Primal-dual interior-point algorithms for second-order cone optimization based on a new parametric kernel function. Acta Math. Sin. Engl. Ser.23, 2027-2042 (2007)
9. Choi, BK, Lee, GM: On complexity analysis of the primal-dual interior-point methods for semidefinite optimization problem based on a new proximity function. Nonlinear Anal., Theory Methods Appl.71, e2628-e2640 (2009) 10. Choi, BK, Lee, GM: On complexity analysis of the primal-dual interior-point method for second-order cone
optimization problem. J. Korean Soc. Ind. Appl. Math.14, 93-111 (2010)
11. Nesterov, YE, Tood, M: Primal-dual interior-point methods for self-scaled cones. SIAM J. Optim.8, 324-364 (1998) 12. Muramatsu, M: On a commutative class of search directions for linear programming over symmetric cones. J. Optim.
Theory Appl.112, 595-625 (2002)
13. Faybusovich, L: Linear systems in Jordan algebras and primal-dual interior-point algorithms. J. Comput. Appl. Math.
86, 149-175 (1997)
14. Faybusovich, L: Euclidean Jordan and interior-point algorithms. Positivity1, 331-357 (1997)
15. Faybusovich, L, Arana, R: A long-step primal-dual algorithm for symmetric programming problems. Syst. Control Lett.
43, 3-7 (2001)
16. Schmieta, S, Alizadeh, F: Extensions of primal-dual interior-point algorithms to symmetric cones. Math. Program.96, 409-438 (2003)
17. Baes, M: Optimization Methods for Convex Symmetric Problems. Monograph, April (2007)
18. Choi, BK, Lee, GM: Complexity analysis for primal-dual interior-point methods for self-scaled optimization problems (submitted)
19. Vieira, MVC: Jordan algebraic approach to symmetric optimization. Electrical Engineering, Mathematics and Computer Science, Delft University of Technology, The Netherlands (2007)
20. Vieira, MVC: Interior-point methods based on kernel functions for symmetric optimization. Optim. Methods Softw.
27, 513-537 (2011)
21. Sun, D, Sun, J: Lowner’s operator and spectral functions in Euclidean Jordan algebras. Math. Oper. Res.33, 421-445 (2008)
22. Faraut, J, Korányi, A: Analysis on Symmetric Cones. Oxford University Press, London (1994)
23. Nesterov, YE, Nemirovskii, A: Interior Point Polynomial Algorithms in Convex Programming. SIAM Stud. Appl. Math., vol. 13. SIAM, Philadelphia (1994)
24. Gowda, MS, Szajder, R, Tao, J: SomeP-properties for linear transformations on Euclidean Jordan algebras. Linear Algebra Appl.393, 203-232 (2004)
25. Faybusovich, L: A Jordan-algebraic approach to potential-reduction algorithms. Math. Z.239, 117-129 (2002) 26. Lim, Y: Applications of geometric means on symmetric cones. Math. Ann.319, 457-468 (2001)
27. Tunçel, L: Potential reduction and primal-dual methods. In: Wolkowicz, H, Saigal, R, Vandenberghe, L (eds.) Handbook of Semidefinite Programming Theory, Algorithms and Applications, pp. 235-265. Kluwer Academic, Boston (2000) 28. Alizadeh, F, Schmieta, S: Symmetric cones, potential reduction methods and word-by-word extensions. In:
Wolkowicz, H, Saigal, R, Vandenberghe, L (eds.) Handbook of Semidefinite Programming, Theory, Algorithms and Applications, pp. 195-233. Kluwer Academic, Boston (2000)
29. Horn, RA, Johnson, CR: Topics in Matrix Analysis. Cambridge University Press, Cambridge (1991)
30. Baes, M: Convexity and differentiability properties of spectral functions and spectral mappings on Euclidean Jordan algebras. Linear Algebra Appl.422, 664-700 (2007)
doi:10.1186/1687-1812-2012-213
