Error Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach

beamer-tu-logo

Error Bound for Classes of Polynomial Systems and its Applications: A Variational

Analysis Approach

Guoyin Li

The University of New South Wales SPOM 2013

Joint work with V. Jeyakumar, B.S. Mordukhovich and T.S. Pham

beamer-tu-logo

Outline

1

Introduction

2

Error Bounds for Convex Polynomials

3

Extensions to Classes of Nonconvex Systems

4

Application to Proximal Point Algorithm

5

Conclusions and Future Work

beamer-tu-logo

Outline

1

Introduction

2

Error Bounds for Convex Polynomials

3

Extensions to Classes of Nonconvex Systems

4

Application to Proximal Point Algorithm

5

Conclusions and Future Work

beamer-tu-logo

Outline

1

Introduction

2

Error Bounds for Convex Polynomials

3

Extensions to Classes of Nonconvex Systems

4

Application to Proximal Point Algorithm

5

Conclusions and Future Work

beamer-tu-logo

Outline

1

Introduction

2

Error Bounds for Convex Polynomials

3

Extensions to Classes of Nonconvex Systems

4

Application to Proximal Point Algorithm

5

Conclusions and Future Work

beamer-tu-logo

Outline

1

Introduction

2

Error Bounds for Convex Polynomials

3

Extensions to Classes of Nonconvex Systems

4

Application to Proximal Point Algorithm

5

Conclusions and Future Work

beamer-tu-logo

For f : R

→ R, we consider the following inequality system (S) f (z) ≤ 0.

To judge whether x is an approximate solution of (S), we want to know d (x , [f ≤ 0]) := inf{kx − zk : f (z) ≤ 0}.

However, we often measure [f (x )]

:= max{f (x ), 0}.

So, we seek an error bound: there exist τ, δ > 0 such that d (x , [f ≤ 0]) ≤ τ [f (x )]

+ [f (x )]

either locally or globally.

beamer-tu-logo

For f : R

→ R, we consider the following inequality system (S) f (z) ≤ 0.

To judge whether x is an approximate solution of (S), we want to know d (x , [f ≤ 0]) := inf{kx − zk : f (z) ≤ 0}.

However, we often measure [f (x )]

:= max{f (x ), 0}.

So, we seek an error bound: there exist τ, δ > 0 such that d (x , [f ≤ 0]) ≤ τ [f (x )]

+ [f (x )]

either locally or globally.

beamer-tu-logo

Definition We say f has a

(1) global error bound with exponent δ if there exist τ > 0 such that d (x , [f ≤ 0]) ≤ τ [f (x )]

+ [f (x )]

for all x ∈ R

(1) (2) local error bound with exponent δ around x if there exist τ,  > 0 such that

d (x , [f ≤ 0]) ≤ τ [f (x )]

+ [f (x )]

for all x ∈ B(x; ) . (2) If δ = 1 in (1) (resp. (2)), we say f has a Lipschitz type global (resp.

local) error bound.

beamer-tu-logo

Error bound is useful in

analyzing the convergence properties of algorithms (e.g. Luo 2000, Tseng 2010 and Attouch etal. 2009);

sensitivity analysis of optimization problem/variational inequality problem (e.g. Jourani 2000)

identifying the active constraints (e.g. Facchinei etal. 1998 and

Pang 1997)

beamer-tu-logo

Some Known Results

Lipschitz type global error bound holds when f is maximum of finitely many affine functions (Hoffman 1951)

Global error bound can fail even when f is convex and continuous (e.g. f (x

, x

) = x

+ q

x

+ x

).

Many further developments (e.g. Ioffe, Kruger, Lewis, Ng, Outrata, Pang, Robinson, Thera etc...)

Global error bound with exponent 1/2 holds when f is a convex

quadratic function. (Luo and Luo, 1994).

beamer-tu-logo

Motivating Example: go beyond quadratic

Consider f (x ) = x

. Then, [f ≤ 0] = {0} and so, d (x , [f ≤ 0]) = |x | ≤ (x

)

= [f (x )]

.

More generally, consider f (x ) = x

with d is an even number. Then,

d (x , [f ≤ 0]) = |x | ≤ (x

)

= [f (x )]

.

beamer-tu-logo

Main Problem

Can we extend the error bound results from convex quadratic

functions to convex polynomials? If yes, how about nonconvex

cases involving polynomial structures?

beamer-tu-logo

What is special about convex polynomials?

Convex polynomial optimization problems can be solved via a sequential SDP approximation scheme (in some cases, one single SDP is enough). (Lasserre 2010 and Jeyakuma and L.

2012).

For a convex polynomial f on R

with degree d , we have (1) inf f > −∞ ⇒ argminf 6= ∅ (Belousov & Klatte 2000);

(2) d (0, ∇f (x

)) → 0 ⇒ f (x

) → inf f (L. 2010);

(3) If f

( v ) = 0, then f (x + tv ) = f (x ) for all x ∈ R

and t ∈ R (Teboulle & Auslender, 2003).

Note: f

(v ) = sup

t

for all x ∈ domf .

beamer-tu-logo

What is special about convex polynomials?

Convex polynomial optimization problems can be solved via a sequential SDP approximation scheme (in some cases, one single SDP is enough). (Lasserre 2010 and Jeyakuma and L.

2012).

For a convex polynomial f on R

with degree d , we have (1) inf f > −∞ ⇒ argminf 6= ∅ (Belousov & Klatte 2000);

(2) d (0, ∇f (x

)) → 0 ⇒ f (x

) → inf f (L. 2010);

(3) If f

( v ) = 0, then f (x + tv ) = f (x ) for all x ∈ R

and t ∈ R (Teboulle & Auslender, 2003).

Note: f

(v ) = sup

t

for all x ∈ domf .

beamer-tu-logo

Let κ(n, d ) = (d − 1)

+ 1.

Theorem (L. 2010)

For a convex polynomial f on R

with degree d . Then there exists τ > 0 such that

d (x , [f ≤ 0]) ≤ τ [f (x )]

+ [f (x )]

for all x ∈ R

. (3) convex quadratic d = 2 (and so, κ(n, d)

= 1/2).

previous example x

n = 1 (and so, κ(n, d)

= 1/d ).

beamer-tu-logo

What is behind the proof?

Łojasiewicz’s inequality and its variants

(Łojasiewicz’s inequality) Let f be an analytic function on R

with f (0) = 0. Then, exists a rational number ρ ∈ (0, 1] and β, δ > 0 s.t. d (x , f

(0)) ≤ β |f (x )|

for all kx k ≤ δ.

(Gwo´zdziewicz 1999) In addition, if f is a polynomial with degree d and 0 is a strict local minimizer, then, ρ =

= κ(n, d )

. Further development on dropping the strict minimizer

assumption in Gwo´zdziewicz’s result (Kurdyka 2012, and L.,

Mordukhovich and Pham 2013).

beamer-tu-logo

Outline of the proof

Induction on the dimension k of [f ≤ 0]

(1) If k = 0, then strict minimizer, so Gwo´zdziewicz’s result can be applied.

(2) Suppose the result is true for k = p;

(3) For the case k = p + 1, find a direction v such that f

(v ) = 0,

and so, f (x + tv ) = f (x ) for all x and for all t. Reduce the case to

k = p.

beamer-tu-logo

Maximum of finitely many convex polynomials?

Extension to maximum of finitely many convex polynomials can fail in general (Shironin, 1986).

Let f

, f

: R

→ R be defined by f

(x

, x

, x

, x

) = x

and f

(x

, x

, x

, x

) = x

+ x

+ x

+ x

x

x

+ x

x

x

+x

x

+ x

x

+ x

x

+ x

x

+ x

+ x

+ x

− x

. Define f = max{f

, f

}. Then global error bound fails for f .

Remark: The implication: f

( v ) = 0 ⇒ f (x + tv ) = f (x )∀x ∈ R

fails

beamer-tu-logo

Maximum of finitely many convex polynomials?

Extension to maximum of finitely many convex polynomials can fail in general (Shironin, 1986).

Let f

, f

: R

→ R be defined by f

(x

, x

, x

, x

) = x

and f

(x

, x

, x

, x

) = x

+ x

+ x

+ x

x

x

+ x

x

x

+x

x

+ x

x

+ x

x

+ x

x

+ x

+ x

+ x

− x

. Define f = max{f

, f

}. Then global error bound fails for f .

Remark: The implication: f

( v ) = 0 ⇒ f (x + tv ) = f (x )∀x ∈ R

fails

beamer-tu-logo

Corollary (L. 2010)

Let f

, i = 1, . . . , m, be nonnegative convex polynomials on R

with degree d

and let d = max

d

. Let f = max

f

. Then there exists a constant τ > 0 such that

d (x , [f ≤ 0]) ≤ τ [f (x )]

+ [f (x )]

−1

+

for all x ∈ R

. (4)

beamer-tu-logo

Classes of nonconvex systems involving polynomial structure Piecewise convex polynomials;

Composite polynomial systems.

beamer-tu-logo

Piecewise convex polynomials

Definition

A function f is said to be a piecewise convex polynomial on R

with degree d if it is continuous and there exist finitely many polyhedra P

, . . . , P

with S

j=1

P

= R

such that the restriction of f on each P

is a convex polynomial with degree d .

Examples: piecewise affine, convex polynomial + αk[Ax + b]

k

.

Can be nonconvex and nonsmooth (e.g. min{x , 1}).

beamer-tu-logo

Example

Consider the piecewise convex polynomial f : R → R defined by f (x ) =

 1 if x ≥ 1, x

if x < 1.

Clearly, [f ≤ 0] = {0}. Now, consider x

= k . Then d (x

, [f ≤ 0]) = k

but f (x

) = 1. So, global error bound fails.

beamer-tu-logo

Notably, in this example, the following implication fails

d (x , [f ≤ 0]) → ∞ ⇒ f (x ) → +∞.

beamer-tu-logo

Theorem (L. 2013)

Let f be a piecewise convex polynomial with degree d . Then, the following statements are equivalent:

(1) d (x , [f ≤ 0]) → ∞ ⇒ f (x ) → +∞.

(2) Global error bound holds with exponent κ(n, d )

, i.e., there exists τ > 0 such that

d (x , [f ≤ 0]) ≤ τ ([f (x )]

+ [f (x )]

−1

+

) for all x ∈ R

. (5)

Remark: (1) is satisfied when f is coercive or when f is convex.

beamer-tu-logo

Composite polynomial systems

Let f (x ) := (ψ ◦ g)(x ) where ψ is a convex polynomial on R

with degree d and g : R

→ R

is a continuously differentiable map.

Theorem (L. & Mordukhovich, 2012)

Let x ∈ [f ≤ 0], and assume that ∇g(x ) : R

→ R

is surjective. Then there exist positive numbers τ and  such that

d x ; [f ≤ 0]) ≤ τ [f (x )]

−1

+

for all x ∈ B

(x , ).

beamer-tu-logo

Applications: Proximal Point Algorithm

Consider the following proximal point algorithm (PPM) for solving min

f (x ):

x

= argmin

{f (x) + 1 2

kx − x

k

}, k = 0, 1, . . . (6)

PPM converges to a solution of min

f (x ) (provided it exists) whenever

∞

X

k =0



= +∞

beamer-tu-logo

Theorem (L. & Mordukhovich, 2012)

Let f be a piecewise convex polynomial on R

with degree d (d ≥ 2).

Suppose that f is convex and inf f > −∞. Let {x

} be generated by the proximal point method (6). Then there exists µ > 0 such that



 



 



d (x

, argminf ) = O

 

1 Pk −1

i=0 _i





if d > 2, d (x

, argminf ) = O

 Q

i=0

√



if d = 2.

(7)

Remark: Can be extended to finding zeros of the maximal monotone

operator T in Hilbert spaces under high-order metric subregularity

condition.

beamer-tu-logo

Conclusion

Error bound is an interesting research topic and has many important applications;

Variational analysis and semi-algebraic techniques could shed

some light on how to improve error bound results from quadratic

to polynomial cases.

beamer-tu-logo

Future Works

Still very preliminary development. A lot of interesting questions, e.g.

(1) Is the derived exponent sharp?

(2) Identify subclasses of convex polynomials s.t. global error bound holds for maximum of finitely many functions within this class?

(3) Local error bound results with explicit exponents for nonconvex polynomials (some partial answer was given in L., Mordukhovich and Pham 2013)?

(4) Any high-order stability analysis for nonconvex polynomial

optimization problems?

beamer-tu-logo

Want to know more?

(1) V. Jeyakumar and G. Li, Duality theory with SDP dual programs for SOS-convex programming via sums-of-squares

representations, preprint 2012.

(2) G. Li, On the asymptotically well behaved functions and global error bound for convex polynomials, SIAM J. Optim., 20 (2010), No. 4, 1923-1943.

(3) G. Li, Global error bounds for piecewise convex polynomials, Math. Program., 137 (2013), 37-64.

(4) G. Li and B.S. Mordukhovich, H ¨older metric subregularity with applications to proximal point method, SIAM J. Optim., 22 (2012), No. 4, 1655-1684.

(5) G. Li, B.S. Mordukhovich and T.S. Pham, New fractional error

beamer-tu-logo

Thanks !

beamer-tu-logo

Let f : R

→ R ∪ {+∞} be defined by

f (x

, x

) =



 

 

x₁²

2x2

, if x

> 0,

0 if (x

, x

) = (0, 0),

+∞ else.

(8)

It can be verified that f is a proper, lower semicontinuous and convex function with inf f = 0. Consider x

= (n, n

). Then one has

f (x

) = 1/2 and ∂f (x

) = ∇f (x

) = (1/n, −1/2n

) → 0.

beamer-tu-logo

f (x

, x

) = x

+ q

x

+ x

. [f ≤ 0] = {(x

, x

) : x

≤ 0, x

= 0}.

Consider x

= (−n, 1). Then d (x

, [f ≤ 0]) = 1 and f (x

) = −n + √

n

+ 1 = √

n²+1+n

→ 0.