Conic Reformulation of Some Quadratic Programming Problems with Applications.

ABSTRACT

TIAN, YE. Conic Reformulation of Some Quadratic Programming Problems with Applications. (Under the direction of Dr. Shu-Cherng Fang.)

Conic Reformulation of Some Quadratic Programming Problems with Applications

by Ye Tian

A dissertation submitted to the Graduate Faculty of North Carolina State University

in partial fulfillment of the requirements for the Degree of

Doctor of Philosophy

Industrial Engineering

Raleigh, North Carolina 2012

APPROVED BY:

Dr. Yahya Fathi Dr. James R. Wilson

Dr. Yunan Liu Dr. Shu-Cherng Fang

DEDICATION

This dissertation is dedicated to my family for their endless love and support: Jiale Tian, my dear Dad

BIOGRAPHY

ACKNOWLEDGEMENTS

I would like to express my deepest and sincerest gratitude to Dr. Shu-Cherng Fang for guiding me through my Ph.D. study at North Carolina State University. His wisdom, knowledge, personality, patience, and professional guidance were invaluable to me.

I am also thankful to Dr. James R. Wilson, Dr. Yahya Fathi and Dr. Yunan Liu for offering valuable comments and suggestions as my committee members. Special thanks to the Graduate School Representative Dr. Negash Medhin for his help in my defense. I am also deeply obliged to Dr. John E. Lavery, Dr. Simon Hisang and Dr. Wenxun Xing for their kind help during my graduate study.

Thanks to the staff of Industrial and Systems Engineering Department, especially Ms. Ce-cilia Chen, for their support and help during my graduate study at NC State. Moreover, thanks to Mr. Edward P. Fitts for the Fitts Fellowship.

I am also grateful to my fellow friends in the Fangroup: Yu-Min Lin, Pingke Li, Kun Huang, Lan Li, Pu Wang, Tao Hong, Lu Yu, Qingwei Jin, Cheng Lu, Yuan Tian, Zhibin Deng, Chia-Chun Hsu, Ziteng Wang, Jian Luo, Chien-Chia Huang, Tiantian Nie, Cheng-Feng Hu, Lihui Zhang, Zhongzhong Jiang, Jingwen Zhang, Chih-Feng Teng, Shih-Han Tseng and many others whose names can not be listed here. Their help and encouragement are always in my mind.

TABLE OF CONTENTS

List of Tables . . . vii

Chapter 1 Introduction . . . 1

1.1 Motivation . . . 1

1.2 Problem Statement . . . 2

1.3 Outline . . . 5

Chapter 2 Literature Review . . . 6

2.1 Mixed Integer Constrained Quadratic Programming Problem . . . 6

2.2 Second-order Cone Constrained Quadratic Programming Problem . . . 7

2.3 Completely Positive Programming Problem . . . 9

2.4 Cardinality Constrained Portfolio Selection Problem . . . 10

Chapter 3 Mixed Integer Constrained Quadratic Programming Problem . . . 13

3.1 Introduction . . . 13

3.2 Lagrangian Dual and Quadratic Reformulation . . . 14

3.3 Solvable Conditions . . . 15

3.4 KKT System and Conic Reformulation Problem . . . 17

3.5 Proposed Algorithm and Numerical Examples. . . 22

3.6 Summary . . . 24

Chapter 4 Second-order Cone Constrained Quadratic Programming Problem 26 4.1 Introduction . . . 26

4.2 Interesting Property of Second-order Cone . . . 27

4.3 Property of Quadratic Function at Infinity . . . 29

4.4 Properties of Conic Reformulation . . . 33

4.5 Exact Formulations . . . 36

4.5.1 Second-order cone with or without a linear constraint . . . 37

4.5.2 Second-order cone with two special linear constraints . . . 46

4.6 Numerical Examples . . . 52

4.7 Summary . . . 54

Chapter 5 Approximation to Completely Positive Programming Problem . . 55

5.1 Introduction . . . 55

5.2 Computable Cone of Nonnegative Quadratic Forms overF_SOC . . . 57

5.3 Approximation to Completely Positive Programming . . . 62

5.4 Adaptive scheme and RLT for improvement . . . 65

5.5 Implementation Issue . . . 68

5.6 Numerical results . . . 70

5.6.1 Box constrained quadratic programming problem . . . 70

5.6.2 Standard quadratic programming problem . . . 72

5.6.4 Binary constrained quadratic programming problem . . . 73

5.7 Concluding remarks . . . 75

Chapter 6 Cardinality Constrained Portfolio Selection Problem . . . 77

6.1 Previous Work . . . 78

6.2 Current Work . . . 79

6.3 Completely Positive Programming Reformulation and Approximation . . . 81

6.4 Approximation Algorithm . . . 84

6.5 Numerical results . . . 86

6.6 Summary . . . 87

Chapter 7 Conclusion . . . 89

7.1 Summary of Dissertation . . . 89

7.2 Future Research . . . 90

LIST OF TABLES

Table 5.1 Results of the algorithm with or without the strategy for initial covering . 70

Table 5.2 Box constrained quadratic programming problems . . . 71

Table 5.3 Standard quadratic programming problems . . . 72

Table 5.4 Maximum clique problems . . . 74

Table 5.5 Binary constrained quadratic programming problems . . . 75

Table 6.1 Portfolio selection problems of Test Set 1 with differentK . . . 87

Chapter 1

Introduction

The aim of this dissertation is to investigate conic reformulations of some important quadrat-ically constrained quadratic programming (QCQP) problems. By using linear conic program-ming, we attempt to explore their optimality conditions, to identify some polynomial-time solvable subclasses, to approximate some hard problems by solving easy problems in quadratic forms, and to apply our new results of QCQP problems to handle some real-life problems.

1.1

Motivation

A quadratically constrained quadratic programming problem to be studied in this dissertation has the following form:

(QCQP)

min f(x) =xTQ0x+bT0x+c0

s.t. xTQix+bTi x+ci ≤0, i= 1, . . . , m1,

xTQix+bTi x+ci = 0, i=m1+ 1, . . . , m1+m2,

x∈_Rn_,

(1.1)

where Qi is an n×n real symmetric matrix, bi is an n-dimensional real vector, ci is a real number,i= 0,1, . . . , m1+m2.

In general, solving a QCQP problem is NP-hard [105]. Therefore, such a problem cannot always be solved in time unless P = NP. However, there exist some polynomial-time solvable subclasses of QCQP problems. By exploring these subclasses, we may develop a deeper understanding of the complicated structures of quadratic optimization. The more computable subclasses can be identified, the more tools can be used for further studies. For a known hard QCQP problem, it is imperative to study its global optimality conditions and design an effective approximation algorithm. Estimating lower bounds of these hard problems by some solvable problems is one of our major interests. The linear conic programming based on the cone of nonnegative quadratic forms or the cone of nonnegative quadratic functions is used to help us study the global optimality conditions, polynomial-time solvable subclasses and approximations to a QCQP problem in this thesis. Last but not least, applying new theoretical results for real practice makes the work meaningful. Therefore, finding suitable applications of QCQP problems becomes an essential part of this dissertation.

1.2

Problem Statement

Many quadratic optimization problems fall in the category of QCQP. In this dissertation, we study three subclasses of QCQP problems which attract attentions in the recent research. The list includes the mixed integer constrained quadratic programming (MIQP) problem, the second-order cone constrained quadratic programming (SOCQP) problem and the completely positive programming (CPP) problem. The MIQP problem and CPP problem are widely adopted in many applications and of great importance in the optimization area. The SOCQP problem is relatively new to the field, but it holds a high potential in both theory and practice. Before giving the statement of these problems, we introduce some notations to be used in the remaining part of this dissertation.

LetF ⊆_Rn_{be a set. The set of interior points ofFis denoted by int(F). The smallest closed} set containingF is the closure ofF denoted by cl(F). We know that int(F)⊆ F ⊆cl(F) and cl(int(F)) = cl(F). Moreover, the boundary of F is defined as bdry(F) = cl(F)\int(F). For a set T ⊆_Rn_{, Cone(T) denotes the smallest cone inR}n_{containing T. The optimal value of an} optimization problem (∗) is denoted byV(∗). LetNdenote the set of positive integers,0∈_Rn denote the vector with all elements being 0,ei∈Rndenote the vector with itsithelement being 1 and others being 0 anden∈_Rn_{denote the vector with all elements being 1. For a scalark∈R,} diag(k) denotes then×ndiagonal matrix with kbeing its diagonal elements. For two vectors

x, y∈_Rn_{,x◦y = (x}

1y1, x2y2, . . . , xnyn)T,kxkis the 2-norm ofx,kxk∞is the∞-norm ofxand Diag(x) denotes then×ndiagonal matrix withxi being itsith diagonal element. Furthermore,

Sn_{denotes the set ofn×nreal symmetric matrices,I}

n∈ Sndenotes then×nidentity matrix,

Nn

matrices,Sn

++ denotes the cone of positive definite matrices. For any U ∈ Sn, U 0 indicates

U ∈ Sn

+,U 0 indicates U ∈ S++n . ForA, B∈ Sn,Aij denotes the element ofAin theith row and jth column, rank(A) denotes the rank of matrix A,A·B = tr(ATB) =Pni=1

Pn

j=1AijBij. For a matrix A ∈ Sn _{with an index set I ⊆ {1, . . . , n}, |I| = m, A}

II ∈ Sm denotes the submatrix ofAwhich takes the elementsAij fori, j ∈I. Witha1∈R, a2∈Rn, SOC(n, a1, a2) denotes the second-order cone {x ∈ _Rn_{| kxk ≤a}

1+aT2x}. With a1 ∈ R, a2 ∈ Rn1, a3 ∈ Rn2, SOC(n1, n2, a1, a2, a3) denotes the second-order cone{(x, y)∈Rn1×Rn2| kxk ≤a1+aT2x+aT3y}. WithM ∈ Sn

++, f ∈Rn,FSOC denotes a general second-order cone{x∈Rn|

√

xT_{M x≤f}T_x}. We let C denote the cone of copositive matrices {A ∈ Sn_{| x}T_{Ax≥0 for all x∈ R}n

+} while C∗ denote the cone of completely positive matrices{X ∈ Sn_|X=Pr

i=1xixTi, xi ∈Rn+, r∈N}. We first study the MIQP problem. It is defined in the following form:

(MIQP)

min f(x) =xT_Qx+bT_x

s.t. xi ∈ {0,1}, i∈I,

xj ∈[0,1], j ∈J,

x∈Rn_,

(1.2)

where Q ∈ Sn _{and b ∈R}n_{, I∪J ={1,2, ..., n} and I ∩J = ∅. If I = ∅, the MIQP problem} becomes the boxed constrained quadratic programming problem. Conversely, if J = ∅, the MIQP problem transforms into the binary constrained quadratic programming problem. By using the extended Lagrangian function on the relaxed box constrained quadratic programming problem, a more general global optimality condition of the original problem can be obtained. However, the reformulated problem may not be convex and solvable. Therefore, in this thesis, we propose a conic reformulation approach which can identify a bigger solvable subclass of the MIQP problem. An effective algorithm is also proposed to return either the optimal value or a lower bound of the MIQP problem. These findings have been documented and published in [122].

Then we study the following SOCQP problem:

(SOCQP) min f(x) =x

T_Qx+bT_x

s.t. x∈SOC(n, a1, a2),

(1.3)

whereQ∈ Sn_,b∈Rn_,a

1 ∈Rand a2 ∈Rn are given.

inequalities (LMI). These results are new to the field.

We also intend to find better lower bounds for the following NP-hard completely positive programming problem:

(CPP)

min f(X) =D·X

s.t. Ai·X =bi, i= 1,2, . . . , m,

X ∈ C∗,

(1.4)

where D ∈ Sn_{, A}

i ∈ Sn, bi ∈ R, i = 1,2, . . . , m, and C∗ is the cone of completely positive matrices. Ifb= (b1, . . . , bm)T, then the dual problem of CPP problem becomes

(DCP)

max h(y) =bTy s.t. Pm

i=1yiAi+S=D,

S∈ C,

(1.5)

whereC is the cone of copositive matrices.

Burer [30] showed that every quadratic optimization problem with linear and binary con-straints can be rewritten as a completely positive programming problem. Therefore, a subclass of QCQP problem is equivalent to the CPP problem. We first provide a computable represen-tation of the cone of nonnegative quadratic forms over a general nontrivial second-order cone. Then, the cone of nonnegative quadratic forms over a union of second-order cones is used to approximate the underlying cone of completely positive matrices in the CPP problem. More-over, an adaptive scheme and Redundant Linear Technique (RLT) constraints are introduced in the algorithm to improve its efficiency.

At last, an application is included to illustrate the practical value of the results obtained. We focus on the so-called “cardinality constrained portfolio selection problem” in the following form:

(PSP)

min f(x) =xTQx

s.t. (en₎T_{x= 1, µ}T_x≥ρ,

|supp(x)| ≤K, xi≥αi,∀i∈supp(x), 0≤xi≤ui, i= 1, . . . , n,

(1.6)

where Q∈ Sn

as the following standard mixed integer quadratic programming problem:

(MIPSP)

min f(x) =xTQx

s.t. (en)Tx= 1, µTx≥ρ,

(en)Ty≤K,

αiyi≤xi ≤uiyi, i= 1, . . . , n,

y∈ {0,1}n_,

(1.7)

This new formulation is a special subclass of the QCQP problem. Since the problem (MIPSP) can be written as a completely positive programming problem, our new approximation scheme can be employed to provide a good lower bound for this problem.

For convenience, in the rest of the dissertation, we either use the abbreviation such as (MIPSP) or use the equation number such as (1.7) to refer to the problem.

1.3

Outline

Chapter 2

Literature Review

In this chapter, we provide a literature review of some major results for the MIQP problem, the SOCQP problem, the CPP problem and the cardinality constrained portfolio selection problem. These results are shown in Section 2.1, Section 2.2, Section 2.3 and Section 2.4, respectively.

2.1

Mixed Integer Constrained Quadratic Programming

Prob-lem

The study of the MIQP problem originated in 1963 [35]. MIQP problems can be found in many industrial and management applications, for example, portfolio optimization [15], economic dispatch of generators [101], multi-vehicle path planning problem [115], quadratic 0-1 knapsack problem [16,17], numerical simulation of friction problem [59] and image reconstruction [80].

In general, a MIQP problem is NP-hard [52]. Identifying polynomial-time solvable subclasses of the MIQP problem and providing different global optimality conditions for these subclasses are important and have been investigated. For example, for the homogeneous binary constrained quadratic programming problem:

min f(x) =xTQx

s.t. x∈ {0,1}n_, (2.1)

the max-cut problem. Sun et al. [121] gave some new results on the duality gap between the binary constrained quadratic programming problem and its Lagrangian dual or semidefinite programming reformulation. They derived a necessary and sufficient condition for the zero duality gap and discussed its relationship with the polynomial-time solvability of the primal problem. Moreover, they provided an effective algorithm to estimate a lower bound of the duality gap. Besides, various algorithms have been proposed to approach the global optimal solutions for the following homogeneous box constrained quadratic programming problem:

min f(x) =xTQx

s.t. 0≤xi ≤1, i= 1, . . . , n.

(2.2)

Dang and Xu [37] designed a barrier algorithm to solve this problem. Yajima and Fujie [127] followed the model of Padberg [100] and developed some valid inequalities to cut the feasible domain. But they can not guarantee that the addition of a finite number of inequalities produces a feasible solution. An and Tao [4] used a branch-and-bound method to relax the box constraints by an appropriate ellipsoid. This algorithm generates a sequence of solutions which converge to the global optimum, but may not converge in a finite number of iterations. Moreover, Hansen et al. [58] developed a finite iteration algorithm using branch-and-bound method on the optimality conditions of the original problem. Burer et al. [30] modeled non-convex quadratic programming programs with mixed binary and continuous variables as linear conic programming problem over the cone of completely positive matrices and proved the equivalence. They also showed the possibility of reducing the dimensions of the completely positive representation and extended the complementarity constraints over bounded variables. Moreover, Bomze et al. [22] interpreted Burer’s work from a topological point of view. They defined a weak key condition and showed that under such condition the Minkowski sum of the lifted feasible set and the lifted recession cone give exactly the closure of Burer’s work.

I will further study the MIQP problem in Chapter 3.

2.2

Second-order Cone Constrained Quadratic Programming

Problem

in-volving sums and maxima of norms, problems with hyperbolic constraints, matrix-fractional problems and SOC-representable functions or sets. Therefore, SOCQP problems are widely applied in engineering, control and finance. Morevoer, some representative works have been done by Lebret [75], Buss et al. [33], Wu et al. [125] and Goldfarb et al. [54,55] in antenna array weight design, grasping force optimization, filter design, image restoration and portfolio selection.

On the algorithmic aspects, although a second-order cone programming (SOCP) problem which has a linear objective function can be solved as a speical case of the SDP problem, the dimension of the corresponding SDP problem increases a lot. Instead, researchers have developed several interior point methods for SOCP problems. Nesterov and Nemirovski [96] proved that a SOCP problem with r second-order cone inequalities can be solved with an iteration complexity of √r by using a self-concordant barrier method. Moreover, Nemirovski and Scheinberg [95] extended the primal and the dual interior point methods developed for linear programming to the SOCP problem. The primal-dual interior point method for SOCP was also studied. Nesterov and Todd [97,98] developed a special primal-dual interior method called NT method and presented their results over self-scaled cones. Meanwhile, Adler and Alizadeh [1] specialized the so-called XZ+ZX method. Monterio and Tsuchiya [90] proved that all methods of the Monteiro-Zhang family have polynomial-iteration complexities.

for the similar problem over the complex field. These exact SDP representations can be solved in polynomial-time [128].

I will further study the SOCQP problem in Chapter 4.

2.3

Completely Positive Programming Problem

Studying the cones formed by quadratic forms that achieve nonnegative values over a given region inRnhas been proven useful in optimization. In general, a cone of nonnegative quadratic forms is not computable. For example, the cone of nonnegative quadratic forms over the nonnegative orthant Rn

+ becomes the cone of copositive matrices.

The study of completely positivity can be traced back to Motzkin [92] in 1952. Many results on copositivity and complete positivity have appeared in linear algebra literatures [13,

62]. In the past two decades, copositive cone and complete positive cones have been used in optimization.

CPP problems are closely related to quadratic optimization problems. D¨ur [41] showed that the standard quadratic programming (StQP) problem can be rewritten as a compositive programming problem. An extension to the multi-StQP problem has been done by Bomze and Schachinger [25]. Moreover, Burer [30] provided a much more general result that every quadratic programming problem with linear and binary constraints can be written as a CPP problem. However, whether a problem with some general quadratic constraints can be transformed into a CPP problem remains an open problem.

Now let us focus on the structure of C∗ _{and C. Both cones are closed, convex, pointed and} non-polyhedral. They are closely related to the cone of positive semidefinite matrices Sn

+ and the cone of nonnegative matrices Nn

+. We have the following relationship

C∗ ⊆ Sn

+∩ N+n and C ⊇ S+n+N+n.

Maxfield and Minc [88] showed that whenn≤4 both inclusions become equality, when n≥5 both inclusions are strict. Most known conditions for completely positivity and copositivity [11, 12, 13, 60, 64] use graph properties or linear algebraic arguments associated with the matrices. These conditions can not be used for algorithmic development in optimization, since the computational task becomes a big issue for large size matrices. Furthermore, Murty and Kabadi [93] proved that checking whether a given matrix is copositive is co-NP-complete. The same computational complexity is needed for checking whether a matrix is completely positive. But for matrices with special structures, such as tridiagonal and acyclic, checking copositivity is a linear-time task [2,19,61].

tech-tions to represent the quadratic form in barycentric coordinates. All strictly copositive matrices are detected as the partition becomes finer. Recently, three new tests were provided by Bomze and Eichfelder [21]. These tests are based on the difference-of-convex (d.c.) decompositions combined with the branch-and-bound method ofw-subdivision type. Sponsel et al. [117] gener-alized the work by using cones betweenNn

+and S+n as certificates. Moreover, many researchers developed their methods directly from the definition of the copositive matrix and the positivity of polynomials. Forr ∈_N, Parrilo [106] defined the hierarchy on cones as follows:

Kr_{={A∈ M}

n|(Pn_i=1Pn_j=1Aijx2ix2j)( Pn

i=1x2i)r has an sum of squares decomposition}. Parrilo proved that Sn

++Nn=K0 ⊂ K1 ⊂ K2 ⊂. . ., and int(C) ⊆ ∪r∈NKr. Therefore, C can

be approximated from the interior. Optimization over Kr _{can be solved using SDP. Similarly,} Bomze and de Klerk [24] defined the following hierarchy of cones:

Cr_{={A∈ M}

n|(Pni=1 Pn

j=1Aijx2ix2j)( Pn

i=1x2i)r has nonnegative coefficients}.

They showed that Nn = C0 ⊂ C1 ⊂ C2 ⊂ . . ., and int(C) ⊆ ∪r∈NCr. Linear programming

applies in optimization over Cr_{. Moreover, Pe˜na et al. [107] developed another hierarchy of} cones Qr _{to approximate C. They actually showed that C}r _{⊆ Q}r _{⊆ K}r _{for all r ∈ N.} Opti-mization overQr _{can be solved by SDP. All of these hierarchies approximate C uniformly, but} the number of LMIs or linear inequalities expands exponentially as r grows. Therefore, with the current SDP-solvers, only low-level approximations are computable. Recently, Bomze et al. [21] proposed a new attempt by using feasible descent method in C∗. They approximated the steepest descent path from each interior point in the process. However, the method is not guaranteed to converge and it requires extra work for finding a feasible starting point together with its factorization. In general, seeking a feasible point is as difficult as solving the original problem.

I will further study the approximation algorithm for the CPP problem in Chapter 5.

2.4

Cardinality Constrained Portfolio Selection Problem

The portfolio selection problem arises from many areas of decision making, such as investment decisions on stocks or bonds, insurance decisions for companies, governments budgeting tex revenues and so on. It becomes an important modeling and analysis tool in modern financial management and investment decision making.

The basic theory of portfolio selection focuses on how to allocate resources among different competing alternatives. The pioneer work was built by Markowitz in his famous article [87]. The author proposed a mean-variance model to measure the expected return and the risk of the portfolio. Suppose there exists a market with n assets and Pi denotes the random return of the ith asset. x = (x1, . . . , xn)T is a variable vector which denotes the weight vector of the portfolio. The total random return of the portfolio is given by Pn

variance of the portfolio return are µTx and xTQx, where µ = (µ1, . . . , µn)T withµi =E(Pi) and Q is ann×n matrix with Qij =E[(Pi−µi)(Pj −µj)]. Then the classical mean-variance model can be expressed in the following form:

(MV)

min f(x) =xTQx s.t. µTx≥ρ,

x∈X,

(2.3)

whereρis a prescribed return level andXis a set formed by other deterministic constraints on

xsuch as budget constraint, sector constraint, no shorting constraint, lower and upper bounds. Based on Markowitz’s work, different extensions of the mean-variance models have been proposed over the past sixty years. Elton and Gruber [43] studied the short sales case which allows some ofxto be negative. Konno and Yamazaki [72] developed a mean absolute deviation (MAD) model which can be solved by linear programming. Young maximized the return of the worst-case scenario in a minimax portfolio selection model in [129]. Then the concepts of Value-at-Risk (VaR) and conditional VaR (CVaR) were introduced as risk measures to evalu-ate the maximum loss under certain confidence level [40, 69, 111, 112]. Li and Ng [77] used a dynamic programming approach to get an optimal solution of multi-period mean-variance portfolio selection. A survey of traditional optimization models for portfolio selection can be found in [104].

Recently, due to the transaction cost and managerial concerns, some real-life trading con-straints are added to the portfolio selection models. In particular, I am interested in the following cardinality constraint, minimum buy-in threshold and maximum hold constraint:

|supp(x)| ≤K and ai ≤xi≤ui, ∀i∈supp(x), (2.4)

mean-[46, 47, 48] proposed a perspective cut and relaxation for a class of convex 0-1 mixed integer program with semi-continuous variables which includes the cardinality constrained quadratic program as a special case. Based on this result, Cui et al. [36] recently proposed a new mixed integer quadratically constrained quadratic programming (MIQCQP) reformulation of the cardinality constrained mean-variance portfolio selection problem whose convex relaxation is tighter than the relaxation of the standard reformulation. Zheng et al. [130] apply a special Lagrangian decomposition scheme to the diagonal decomposition of the problem. This leads to an SDP formulation for computing the “best” diagonal decomposition in the perspective reformulation. Many other authors also studied some heuristic and local search methods in the portfolio selection models with cardinality constraints and minimum threshold in the context of limited-diversification, small portfolios and empirical study with real features [18,34,67,86,89].

Chapter 3

Mixed Integer Constrained

Quadratic Programming Problem

In this chapter, we focus on the following MIQP problem:

(MIQP)

min f(x) =xTQx+bTx s.t. xi ∈ {0,1}, i∈I,

xj ∈[0,1], j ∈J,

x∈_Rn_,

(3.1)

whereQ∈ Sn_,b∈Rn_{,I∪J ={1,2, ..., n}andI∩J =∅. For this problem, some more general} global optimality conditions, a larger solvable subclass, and an practical algorithm are studied in this chapter.

3.1

Introduction

In general, the MIQP problem is NP-hard, i.e., there is no polynomial-time algorithm for solving it, unless P=NP. However, quadratic and conic reformulations may be used to identify a polynomial-time solvable subclass of MIQP problems. Notice that, if an optimization problem with hidden convexity can be formulated into an equivalent problem with a unique local optimal solution, then an effective algorithm combining any conventional local-search methods can be designed for the global optimality. Some conditions for the MIQP problem to be transformed into different solvable reformulations will be discussed later.

practical algorithm is provided with two numerical experiments. At last, we give a summary in Section 3.6.

3.2

Lagrangian Dual and Quadratic Reformulation

LetG ={λ∈_Rn_|λ

j ≥0 for j∈J}, then the Lagrangian function for MIQP is defined as

L(x, λ) = 1₂xTQx+bTx+P

i∈Iλi(x2i −xi) +Pj∈Jλj(x2j −xj)

= 1₂xT(Q+ 2Λ)x+ (b−λ)Tx, (3.2)

whereλ∈ G and Λ = Diag(λ).

Since a general MIQP problem is nonconvex, the duality gap between the MIQP problem and its conventional Lagrangian dual problem is not always zero. Hence we consider the extended Lagrangian dual function defined as follows:

Pe_{(λ) = min}

x∈[0,1]nL(x, λ). (3.3) Let F(MIQP) denote the feasible domain of problem (MIQP). Notice that, the value of the extended Lagrangian dual function is always smaller than the optimal value of problem (MIQP), which is stated in the next theorem.

Theorem 1. Pe(λ)≤V(MIQP)for any λ∈ G.

Proof. Since F(MIQP) ⊂ [0,1]n, by definition, we have Pe(λ) ≤ minx∈F(MIQP)L(x, λ). For any x ∈ F(MIQP), it is easy to verify that L(x, λ) = ₂1xTQx +bTx +P

i∈Iλi(x2i −xi) + P

j∈Jλj(x2j −xj) = 1₂xTQx+bTx +Pi∈Jλj(x2j −xj). Since λj ≥ 0 and x2j −xj ≤ 0 for any j ∈ J, then L(x, λ) ≤ f(x) for any x ∈ F(MIQP). Hence Pe(λ) ≤ minx∈FL(x, λ) ≤

minx∈Ff(x) =V(MIQP).

The extended Lagrangian dual problem is defined as

(ELD) max{λ∈G}Pe(λ). (3.4)

Unlike the conventional Lagrangian dual problem, the extended Lagrangian dual problem always satisfies the strong dual principle.

Theorem 2. The gap between problem (ELD)and problem (MIQP)is always zero.

Proof. Theorem 1 indicates thatPe(λ)≤V(MIQP) for anyλ∈ G . Thus, we only need to show the equality is attainable. Choose a ¯λ∈_Rn_{satisfying (Q+ 2 ¯Λ)II ≺0 and ¯λ}

extended Lagrangian dual functionPe(λ). For anyi∈I, letgi(t) =L(¯x+tei,¯λ) be a univariate quadratic function defined on t ∈ [0−x¯i,1−x¯i], then we have d

2_gi(t)

dt2 = eTi (Q+ 2 ¯Λ)ei < 0. Hence gi(t) is strictly concave on t ∈[0−x¯i,1−x¯i] and its minimizer can be attainable only at t = 0−x¯i or t = 1−x¯i. As assumed, ¯x is an optimal solution for minx∈[0,1]nL(x,¯λ), which implies that t = 0 is a minimizer of gi(t). Consequently, ¯xi = 0 or ¯xi = 1. Moveover,

Pe(¯λ) = minx∈[0,1]nL(x,¯λ) = min_x∈F(MIQP)L(x,λ¯) = min_x∈F(MIQP) f(x) = V(MIQP). This shows the gap between problem (ELD) and problem (MIQP) is zero.

Let λ∗ be an optimal solution of problem (ELD), we define a new problem as follows:

(RQP) min L(x, λ

∗₎

s.t. xi ∈[0,1], i= 1,2, ..., n.

(3.5)

Based on Theorem 2, we have the next corollary.

Corollary 3. If λ∗ be an optimal solution of problem (ELD), then problem (RQP) is a re-formulation of problem (MIQP) in the sense that any optimal solution of problem (MIQP) is optimal to problem (RQP).

Proof. Let x∗ be an optimal solution of problem (MIQP). The definition of the extended Lagrangian function Pe(λ) leads to Pe(λ∗) = minx∈[0,1]nL(x, λ∗) ≤ L(x∗, λ∗) ≤ f(x∗) =

V(MIQP). Besides, Theorem 2 has shown thatPe(λ∗) =V(MIQP). Hence minx∈[0,1]nL(x, λ∗) =

L(x∗, λ∗),x∗ is optimal to problem (RQP).

This corollary implies that the Lagrangian function L(x, λ∗) with λ∗ being an optimal solution of problem (ELD), is a quadratic reformulation of problem (MIQP). This reformulation transforms the original problem into a continuous optimization problem. However, there may exist multiple optimal solutions of problem (ELD) and lead to many reformulations. Our object is to find a solvable reformulation for problem (MIQP).

3.3

Solvable Conditions

The solvability of the reformulation for problem (MIQP) is discussed in this section. One obvious solvable case is the convex case, i.e., if L(x, λ∗) is a convex function with an optimal solution λ∗ of problem (ELD), then problem (RQP) is solvable.

Proof. IfQ+ 2Λ∗ 0, then problem (RQP) is a convex quadratic programming problem, which is polynomial-time solvable [110]. Besides, ifQ+2Λ∗ 0, then the objective function of problem (RQP) is strictly convex and there exists a unique optimal solution of problem (RQP).

The “Convex Solvable Condition” is the simplest solvable condition of problem (MIQP). From the view of nonlinear programming (NLP), any optimal solutionx∗ of problem (MIQP) must satisfy the following KKT condition, that is, there exists a Lagrangian multiplier vector

λ∗∈Rn _{such that}

(KKT System)

∇xL(x, λ∗)|x=x∗ = (Q+ 2Λ∗)x∗+b−λ∗ = 0,

x2

j −xj ≤0, λ∗j ≥0, forj∈J,

x2_i −xi= 0, fori∈I,

λ∗_i(x2_i −xi) = 0, i= 1,2, ..., n.

(3.6)

In the literature, we can find the following result:

Condition 5. (Positive Semidefinite Condition[44]) Let(x∗, λ∗)be a KKT pair of problem (MIQP)satisfying Q+ 2Λ∗0.

The relationship between two conditions is stated in the following theorem:

Theorem 6. If (x∗, λ∗) is a KKT pair of problem (MIQP) satisfying Q+ 2Λ∗ 0, then λ∗

satisfies the “Convex Solvable Condition”.

In this way, the Lagrangian multipliers in the “Positive Semidefinite Condition” can be interpreted as a convex reformulation of problem (MIQP).

Now a solvable condition for a nonconvex reformulation is considered. Let (x∗, λ∗) be a KKT pair of problem (MIQP). Define the tangent cone T(x∗) atx∗ asT(x∗) ={d∈_Rn_|0≤

di ifx∗i = 0; di ≤0 if x∗i = 1}. We show the global optimality condition of problem (MIQP) and problem (ELD).

Theorem 7. Let (x∗, λ∗) be a KKT pair of problem(MIQP). Ifλ∗ satisfiesdT(Q+ 2Λ∗)d≥0 for all d∈T(x∗), then x∗ is a global optimal solution to both of problem (MIQP) and problem (RQP). Moreover, λ∗ is an optimal solution of problem (ELD).

Proof. For any x∈[0,1]n_{, let d=x−x}∗ _{and f}

d(t) =L(x∗+td, λ∗) be a univariate quadratic function ont. It is easy to verify thatd∈T(x∗). Then, for anyt∈[0,1], we havex∗+td∈[0,1]n. The functionfd(t) satisfies d2_dtfd(t)2 =dT(Q+2Λ

∗_{)d≥0 and} dfd(t)

dt |t=0=dT[(Q+2Λ

principle, we have V(ELD)≤V(MIQP). Hence Pe(λ∗) = f(x∗) = V(MIQP), x∗ is optimal to problem (MIQP), andλ∗ is optimal to problem (ELD).

Theorem 7 presents an optimality condition for problem (MIQP) and problem (ELD). Recall that, if there exists a KKT pair (x∗, λ∗) satisfying the condition in Theorem 7, then problem (RQP) is a reformulation for problem (MIQP). We propose a condition for the solvability of the reformulated problem in the next theorem.

Theorem 8. Let (x∗, λ∗) be a KKT pair of problem (MIQP). If dT(Q+ 2Λ∗)d > 0 for all

d∈T(x∗)\ {0}, then x∗ is the unique local optimal solution of problem (RQP).

Proof. The proof is similar to that of Theorem 7. For any x ∈ [0,1]n such that x 6= x∗, let

d= x−x∗ and fd(t) be defined as in Theorem 7. Following a similar method, we can prove

fd(1)> fd(0) andfd(t) is a strictly increasing function on [0,1]. Therefore,L(x, λ∗)> L(x∗, λ∗) for all x ∈ [0,1]n, x 6= x∗ and x is not a local optimal solution of L(x, λ∗) (since −d is a decreasing feasible direction at x for the reformulated problem). Hence x∗ is the unique local optimal solution to problem (RQP).

Theorem 8 proposes a new sufficient condition for the solvability of the reformulated prob-lem. Notice that, the reformulation may not be convex. However, since it has only one local optimal solution, it can be solved by any local optimization algorithm. This new global opti-mality condition is more general than the “Positive Semidefinite Condition”.

Condition 9. (Second Order Solvability Condition) Let(x∗, λ∗)be a KKT pair of problem (MIQP) satisfying dT(Q+ 2Λ∗)d≥0 for all d∈T(x∗), then we say problem (MIQP)satisfies the Second Order Solvability Condition. If dT_{(Q+ 2Λ}∗_{)d > 0 for all d ∈ T(x}∗_{), then we say} problem (MIQP) satisfies the Second Order Strong Solvability Condition.

I.M.Bomze [20] provided a global optimality condition for the quadratic programming prob-lem with a polyhedron feasible set. Our “Second Order Solvability Condition” of probprob-lem (MIQP) is similar to that result. In this chapter, we will not only provide a solvability condi-tion in theory, but also a method to build the reformulacondi-tion in practice.

3.4

KKT System and Conic Reformulation Problem

Recall that, for a given setF ⊆ Rn_{, Sturm and Zhang [120] defined the cone of nonnegative} quadratic functions over F as follows:

DF = (

M ∈ Sn+1 _| " 1 x #T M " 1 x #

≥0,∀x∈ F

)

. (3.7)

Therefore, DF is the set of the coefficients of quadratic functions which are nonnegative over the feasible domain of F. The dual cone of DF is

D∗_F = cl Cone ( X= " 1 x # " 1 x #T

|x∈ F

)

. (3.8)

DF and D∗F are both closed and convex.

Now, we define the following cone on the box constrained domain [0,1]n:

Dbox = (

U ∈Mn+1 | " 1 x #T U " 1 x #

≥0,∀x∈[0,1]n )

.

Moreover, for a KKT pair (x∗, λ∗), define the matrix

D(x∗, λ∗) = "

−f(x∗) 1₂(b−λ∗)T 1

2(b−λ ∗₎ 1

2Q+ Λ ∗

#

.

Now an important relationship between a KKT pair and the “Second Order Solvable Condition” can be depicted.

Theorem 10. Let (x∗, λ∗) be a KKT pair of problem (MIQP). Then, (x∗, λ∗) satisfies the “Second Order Solvable Condition” if and only if D(x∗, λ∗)∈Dbox.

Proof. Since x∗ is a KKT solution of problem (MIQP), we have x∗_i = 0 or 1 for all i∈I and 0≤x∗_i ≤1 for all i∈J. For anyx∈[0,1]n, let d=x−x∗. It is easy to verify that d∈T(x∗). And for anyd∈T(x∗), we multiply a positive scale to make a newdsatisfy that 0≤x∗+d≤1. Then we have that

" 1

x

#T "

−f(x∗) 1₂(b−λ∗)T 1

2(b−λ ∗₎ 1

2Q+ Λ ∗ # " 1 x # = " 1

x∗+ ¯d

#T"

−f(x∗) 1₂(b−λ∗)T 1

2(b−λ ∗₎ 1

2Q+ Λ ∗

# " 1

x∗+ ¯d

#

=−f(x∗) +L(x∗, λ∗) + ¯dT((Q+ 2Λ∗)x∗+b−λ∗) + ¯dT(Q+ 2Λ∗) ¯d

Notice that, ¯dT(Q+ 2Λ∗) ¯d≥0 if and only if dT(Q+ 2Λ∗)d≥0. Therefore,dT(Q+ 2Λ∗)d≥0 for all d∈T(x∗) if and only ifD(x∗, λ∗)∈Dbox.

The above theorem transforms the “Second Order Solvable Condition” to the condition of D(x∗, λ∗) ∈ Dbox. Denote D∗box = cl Cone

(

X= "

1

x

# " 1

x

#T

|x∈[0,1]n )

as the dual cone of

Dbox. We define the following linear conic programming problem:

(COP)

min "

0 1₂(b−λ)T 1

2(b−λ) 1 2Q+ Λ

#

·Y

s.t.

" 1 xT

x X

# =Y, Xii−xi = 0, i∈I,

Xjj−xj 60, j ∈J,

Y ∈D∗_box.

(3.9)

Its dual problem becomes

(COD)

max −σ

s.t.

"

σ 1₂(b−λ)T 1

2(b−λ) 1 2Q+ Λ

#

∈Dbox,

λj ≥0,∀j∈J.

(3.10)

Since F(MIQP) is bounded, the optimal value of problem (MIQP) is finite. We have the following theorem for the relationship among problems (MIQP), (COP) and (COD).

Theorem 11. Problems (MIQP),(COP) and (COD)share the same optimal value.

Proof. Suppose x∗ is an optimal solution of problem (MIQP). By letting X∗ = x∗(x∗)T, we see that it is easy to verify that (x∗, X∗) is a feasible solution of problem (COP), hence we haveV(COP)≤V(MIQP). Besides, the weak duality theorem of linear conic programming [81] implies thatV(COD)≤V(COP). We would like to show thatV(COD)=V(MIQP). Note that

"

σ 1₂(b−λ)T 1

2(b−λ) 1 2Q+ Λ

#

∈Dbox

" 1

x

#T"

σ 1₂(b−λ)T 1

2(b−λ) 1 2Q+ Λ

# " 1

x

#

≥0,∀x∈ F.

This is equivalent to −σ ≤ L(x, λ) for all x ∈ F(MIQP). Let ¯λ = 0 and −σ¯= V(MIQP)= minx∈F(MIQP) f(x) = minx∈F(MIQP)L(x,λ¯). Then (−σ,¯ λ¯) is a feasible solution of problem (COD) with V(COD)≥ −σ = V(MIQP). Therefore, we have that V(MIQP)= V(COP)=

V(COD).

Lu et al. [83] proved a similar conclusion for a QCQP problem. Since the optimal value of problem (MIQP) and (COD) are the same, we can check the relationship of their optimal solutions.

Theorem 12. Let (x∗, λ∗) be a KKT pair of problem (MIQP). If the corresponding matrix

D(x∗, λ∗) ∈ Dbox, then x∗ is an optimal solution of the problem (MIQP) while (σ∗, λ∗) is an optimal solution of problem (COD) with σ∗ =−f(x∗).

Proof. Let X∗ = x∗(x∗)T and Y∗ = "

1

x∗

# " 1

x∗

#T

. Since x∗ is a KKT solution, it is easy to verify that Y∗ ∈D_box∗ and D(x∗, λ∗)·Y∗ =−f(x∗) +L(x∗, λ∗) = 0. Thus the complementary condition is met. From the optimality theory of linear conic programming [83], we know that (x∗, X∗) is a global optimal solution of problem (COP), and (σ∗, λ∗) is an optimal solution of problem (COD) with σ∗=−f(x∗). Since (x∗)TQx∗ =−σ∗ and problems (MIQP), (COP) and (COD) share the same optimal value,x∗ is an optimal solution of problem (MIQP).

Notice that, the above theorem also provides a sufficient condition of global optimality for problem (MIQP).

Condition 13. (Extended Global Optimality Condition) Let (x∗, λ∗) be a KKT pair of problem (MIQP) satisfyingD(x∗, λ∗)∈Dbox.

However, there is no guarantee to obtain an expected λ∗ by solving problem (COD). The property of λ∗ needs to be explored further.

Lemma 14. Let (x∗, λ∗) be a KKT pair of problem (MIQP) satisfying D(x∗, λ∗) ∈ Dbox. If (σD, λD) is an optimal solution of problem (COD), then λD ≤λ∗.

Proof. Let Y∗ = "

1

x∗

# " 1

x∗

#T

V(MIQP)= −σ_D ≤

"

σD 1₂(b−λD)T 1

2(b−λD) 1

2Q+ ΛD #

·Y∗−σD = L(x∗, λD) ≤ f(x∗) = V(MIQP).

Since "

, σD 1₂(b−λD)T 1

2(b−λD) 1

2Q+ ΛD #

∈Dbox, for any x∈[0,1]n, we have

" 1

x

# "

σD 1₂(b−λD)T 1

2(b−λD) 1

2Q+ ΛD # "

1

x

#T

≥0.

HenceL(x, λD)≥ −σD =V(MIQP) for anyx∈[0,1]n. Therefore,x∗is optimal for the problem minx∈[0,1]n L(x, λ_D). Notice that the gradient of the functionL(x, λ_D) atx∗ is (Q+ 2Λ_D)x∗+ (c−λD). If [x∗]i= 1, then −ei is a feasible direction forL(x, λD) at x∗. If [x∗]i = 0, thenei is a feasible direction for L(x, λD) at x∗. Because x∗ is a global optimality solution for problem (RQP), , its optimality condition implies thatdT[(Q+ 2ΛD)x∗+ (b−λD)]≥0, wheredis any feasible direction atx∗. Therefore, if [x∗]i = 1, then (Q+ 2ΛD)x∗+ (b−λD)≤0, If [x∗]i = 0, then (Q+ 2ΛD)x∗+ (b−λD)≥0. Letr = [(Q+ 2ΛD)x∗+ (b−λD)]−[(Q+ 2Λ∗)x∗] + (b−λ∗)]. From KKT condition, (Q+2Λ∗)x∗]+(b−λ∗_{) = 0. Moreover, ifx}∗_{= 0 thenr}

i ≥0, ifx∗ = 1 then

ri ≤0. Noticingri= (4x∗i −2)[(λD−λ∗)]i, we know that whenever x∗i = 1 or 0, [λD]i ≤[λ∗]i. For any 0 < x∗_i < 1, from KKT condition, we have that λi = 0. Since L(x, λD) = f(x∗, [λD]i[(x∗i)2−x∗i] = 0, for any 0< x∗i <1, [λD]i= 0. Therefore,λD ≤λ∗.

Due to the special property of λ∗, we have the following key result:

Theorem 15. Let (x∗, λ∗) be a KKT pair of problem (MIQP)satisfying D(x∗, λ∗)∈Dbox. If (σD, λD) is an optimal solution of the problem (COD), we have thatσD =−V(MIQP) andλ∗ is the unique optimal solution of the following linear conic programming problem:

max eTλ s.t.

"

σD 1₂(b−λ)T 1

2(b−λ) 1 2Q+ Λ

#

∈Dbox,

λi ≥0,for alli∈J.

(3.11)

Proof. Notice that any feasible solution (σD, λ) of problem (3.11) is optimal to problem (COD) withλ≤λ∗. Because (σD, λ∗) is feasible to problem (3.11) and the constraints forxi are linear independent,λ∗ is the unique optimal solution of problem (3.11).

3.5

Proposed Algorithm and Numerical Examples.

Recall that, problems (MIQP) and (COD) share the same optimal value, so the whole difficulty of problem (MIQP) is packed into the coneDbox. Since there is no polynomial-time algorithm for solving problem (COD), we need to find a computable coneCF such thatCF ⊆Dbox, to sub-stitute the coneDbox. Given such a coneCF, we define the following linear conic programming problem:

max σ

s.t.

"

−σ 1

2(b−λ) T 1

2(b−λ) 1 2Q+ Λ

#

∈CF,

λj ≥0,for any j∈J.

(3.12)

Assumingσdis the optimal solution of problem (3.12), an additional linear conic programming problem is defined as follow:

max eTλ s.t.

"

σd 1₂(b−λ)T 1

2(b−λ) 1 2Q+ Λ

#

∈CF,

λj ≥0,for any j∈J.

(3.13)

Correspondingly, we have the next result.

Theorem 16. Let (x∗, λ∗) be a KKT pair of problem (MIQP) and CF be a computable cone such that such that CF ⊆ Dbox. If the corresponding matrix D(x∗, λ∗) ∈ CF, then λ∗ is the unique optimal solution of problem (3.11).

Proof. Since D(x∗, λ∗) ∈ CF ⊆ Dbox, we know that the σd in problem (3.13) is equal to

V(MIQP). Besides, by Theorem 15, λ∗ is optimal for problem (3.11) and feasible for problem (3.13). Moreover, any feasible solution of problem (3.13) is also feasible for problem (3.11),λ∗

must be the unique optimal solution of problem (3.13).

Remark 17. The primary purpose of this chapter is to provide a more general global optimality condition and identify a bigger solvable subclass of theMIQPproblem. We do not aim to discuss how to find a computable cone CF to approximate the coneDbox. This is a big issue in recent research [28,74, 106,131]. The performance of the lower bound is determined by the choice of

CF. A tighter inner approximation of the cone Dbox provides a better lower bound.

With problem (3.12) and problem (3.13) being defined, the following algorithm is designed to compute a reformulation:

Algorithm 1. MIQP Algorithm:

a computable cone CF.

Step 2: Solve problem (3.12) to get its optimal value σd.

Step 3: Construct the linear conic relaxation problem (3.13) using σd and CF.

Step 4: Solve the problem (3.13) to get its optimal value λ∗ and the reformulation L(x, λ∗). Step 5: Solve problem(RQP) withλ∗ to obtain a local optimal solution x∗. Iff(x∗) =σd, then return x∗ as an optimal solution of the MIQPproblem. Otherwise, stop, the algorithm returns a lower bound of the MIQPproblem.

Remark 18. To the best of our knowledge, the known largest solvable subclass of the MIQP problem is the subclass satisfying the “Positive Semidefinite Condition” with Q+ 2Λ being invertible. If CF = S₊n+1, these two conditions become equivalent. Whereas, the cone CF can be better chosen based on the structure of the feasible domain F. For example, a bigger cone

CF = S+n+1+N+n+1. In this way, the performance of Algorithm 1 could be improved. In this sense, our result extends the known solvable subclass to a larger one.

The next two examples show that Algorithm 1 indeed works.

Example 19.

min 1 2x

T_Qx+cT_x

s.t. x∈ {0,1}n_.

with

Q= 

           

417 716 −4 732 −69 478 40 716 919 73 1131 −71 104 −37

−4 73 161 54 −182 11 49

732 1131 54 123 −25 620 −49

−69 −71 −182 −25 412 16 −133

478 104 11 620 16 353 34

40 −37 49 −49 −133 34 159 

           

and b=

h

−3 58 −56 57 −90 −123 −34 iT

.

Here we choose CF = S₊n+1 +Nn+1. Using Algorithm 1, we have the optimal solution

Example 20.

min 1 2x

T_Qx+cT_x

s.t. xi ∈[0,1], i= 1,· · · , n

with

Q= 

                   

263 −97 62 217 52 621 935 258 −61 −10

−97 299 −17 9 −4 −123 −17 −40 −3 37

62 −17 178 71 −118 −83 −110 9 −56 42

217 9 71 143 −5 842 228 42 58 −41

52 −4 −118 −5 177 102 −15 120 13 −52 621 −123 −83 842 102 219 574 22 73 −53 935 −17 −110 228 −15 574 457 154 −25 84

258 −40 9 42 120 22 154 473 18 −29

−61 −3 −56 58 13 73 −25 18 −4 −79

−10 37 42 −41 −52 −53 84 −29 −79 224



                   

b=h−20 −314 46 −83.45 −128.7 41.3 43.85 −341.8 −34.05 −34.6 iT

.

Here we choose CF = S+n+1 +N+n+1. Using Algorithm 1, we have the optimal solution

λ∗ = [41.1732 35.1504 0 20.0594 0 13.1491 70.1486 0 88.0332 0]T. Notice that, Q+ 2Λ∗ is not positive semidefinite and problem (RQP) is not convex. However, solving the reformulation by some local search methods, we can obtain a global optimal solution

x∗ = [0.0001 0.9996 0.5 0 0.7501 0 0.0001 0.5999 0.9998 0.5]T.

These two examples demonstrate that Algorithm 1 is applicable for a larger solvable subclass of the MIQP problem. Based on different structures of some given problems, we may find some tighter computable cones to improve the performance of our algorithm.

3.6

Summary

Chapter 4

Second-order Cone Constrained

Quadratic Programming Problem

In this chapter, we focus on the following SOCQP problem:

(SOCQP) min f(x) =x

T_Qx+bT_x

s.t. x∈SOC(n, a1, a2),

(4.1)

whereQ∈ Sn_{and b∈R}n_{. An interesting property of the SOCQP problem will be shown and} analyzed. Then, some computable conic reformulations for SOCQP problems based on exact representations of the cones of nonnegative quadratic functions over different second-order cone domains will be given.

4.1

Introduction

Second-order cone is a simple unbounded domain. If the objective function is linear, the SOCQP problem can be simplified as an second-order cone programming (SOCP) problem. In general, SOCQP problems can be written as a subclass of QCQP problems with two constraints. To the best of our knowledge, for an optimization problem with a general nonconvex quadratic objective function over a domain of, only the following can be polynomial-time solved.

• F ={x∈_Rn_|xT_Q

1x+ 2bT1x+c1 ≤0},

• F ={x∈_Rn_|xT_Q

1x+ 2bT1x+c1 = 0} withQ1 being strictly negative or positive definite,

• F ={x∈_Rn_|xT_Q

1x+ 2bT1x+c1 ≤0, and aTx≤a0} withQ1 being positive semidefinite,

• F = {x ∈ _Rn_{| x}T_Q

1x+ 2bT1x+c1 ≤ 0, xTQ1x+ 2bT2x+c2 ≤ 0} with Q1 being positive semidefinite,

• F ={x| kxk ≤1, l≤aTx≤u} withl < u.

unbounded second-order cones of SOC(n, a1, a2),SOC(n1, n2, a1, a2, a3) and the following two second-order cone domains:

• F₁ ={(x, y)∈_Rn1 _×Rn2_{|kxk ≤a}

1+aT2x+aT3y, a1+aT2x+aT3y≥a4 ≥0},

• F₂ ={(x, y)∈_Rn1 _×Rn2_{|kxk ≤a}

1+aT2x+aT3y, a5≥a1+aT2x+aT3y≥a4≥0}.

Notice thatF1,F2are defined on the second order cone SOC(n1, n2, a1, a2, a3) and SOC(n, a1, a2) is a special case of SOC(n1, n2, a1, a2, a3).

The layout of this chapter is as follows. Section 4.2 shows a special structure of the second-order cone and an interesting property of the quadratic objective function at infinity in the second-order cone. Section 4.3 analyzes that property in detail. Section 4.4 explores a linear conic reformulation of the SOCQP problem. Section 4.5 provides some exact representations of the cone of nonnegative quadratic functions over different second-order cone domains. Several numerical examples are provided in Section 4.6. At last, we give a summary in Section 4.7.

4.2

Interesting Property of Second-order Cone

For an optimization problem, we usually have a wrong impression that if the optimal value is finite, the corresponding optimal solution must be finite. However, an interesting property of the second-order cone domain shows that optimal solutions of problem (SOCP) may be at infinity, however, as the variable approaches these optimal solutions, the objective value may converge to a finite optimal value.

In this section, a special case of the second-order cone SOC(n, a1, a2), SOC(n) = {x ∈

Rn| qx2

1+x22+. . .+x2n−1 ≤xn}={x∈Rn| kxk ≤aTx}with aT = (0,0, . . . ,

√

2), is used to show the interesting property. The corresponding SOCQP problem can be written as following:

min f(x) =xTQx+bTx

s.t. x∈SOC(n). (4.2)

Now we define the cross-section St of SOC(n):

St={x∈Rn| n−1 X

i=1

x2_i ≤x2_n, xn=t, t >0}. (4.3)

The next lemma shows an important structure of the cross-sectionSt.

Lemma 21. For any x ∈ SOC(n) and a fixed positive number t, we can find a point x˜ ∈ St such that x=λx˜, λis a nonnegative scalar.

let ˜x = _xnt x and λ = xn_t . It is easy to check that ˜x ∈ St, λ > 0 and x = λx˜. Therefore, for any point in the second-order cone SOC(n), we can find a point in a fixed cross section St to represent it.

Lemma 21 implies that cross-sections of the second-order cone may contain some useful information to draw a rough picture about the problem.

Based on the general quadratic objective function f(x) and a point ¯x in the fixed cross-section Stwith t >0, we define a new function hx¯(λ) :R+→Ras following:

hx¯(λ) =f(λx¯) = ¯xTQxλ¯ 2+ 2bTxλ.¯ (4.4)

hx(¯ λ) is a univariate quadratic function on λ with ¯xTQx¯ as the coefficient of the quadratic term. Define{x∈_Rn+1_{|x=λx, λ¯ ≥0} to be a halfline of ¯xin the second-order cone SOC(n).} Then, as we movexalong the halfline of ¯x to infinity, the following three situations need to be discussed:

• x¯TQx >¯ 0, hx(¯ λ) is a convex function, limλ→∞h¯x(λ)→ ∞.

• x¯TQx <¯ 0, hx(¯ λ) is a concave function, limλ→∞hx(¯ λ)→ −∞.

• x¯TQx¯= 0, hx(¯ λ) is a linear function,

(i) limλ→∞hx¯(λ)→ ∞,ifbTx >¯ 0, (ii) limλ→∞hx(¯ λ)→ −∞,ifbTx <¯ 0, (iii) limλ→∞hx¯(λ) = 0,ifbTx¯= 0.

Based on the discussion above, we know that if there exists a point ¯x∈St such that ¯xTQx <¯ 0 or ¯xTQx¯= 0, bTx <¯ 0, problem (4.2) will take −∞ as its optimal value and optimal solutions will be infinite. Let A = {x ∈ St| xTQx = 0} denote the set of all points in St which satisfy xTQx = 0 (A can be an empty set). If xTQx ≥ 0 for all x ∈ §_t and bTx > 0 for all

x∈ A, problem (4.2) will take the optimal solution at original point and the optimal value is 0. However, for the remaining special situation,∃x¯∈Stsuch that ¯xTQx¯= 0, bTx¯= 0,V(SOCP) may be infinite or finite with infinite optimal solutions. Next, we provide two small examples to show this property.

Example 22. Consider the following two problems:

inf f1(x) =xTQx+bT₁x

inf f2(x) =xTQx+bT₂x

s.t. x∈SOC(4), (4.6)

where b1 = (−2,2,2,2)T, b2 = (2,2,2,2)T, SOC(4) ={x∈R4| p

x2₁+x2₂+x2₃ ≤x4},

Q= 

    

1 1 0 0

1 1 0 0

0 0 1 1

0 0 1 1



    

.

Q is positive semidefinite and there does not exist any x ∈SOC(4) such that xTQx = 0, bTx < 0, but, we can find a x ∈ SOC(4) such that xTQx = 0, bTx = 0, for example x = (0,0,−1,1)T_.

For problem (4.5), V(4.5) =−∞and the corresponding optimal solutionx∗ is at infinity. For problem (4.6), V(4.6) = −1, however, the corresponding optimal solution x∗ is also at infinity, x∗ = limM→∞(−0.5,−0.5,−M,√M2_{+ 0.5).}

Remark 23. Notice that, in Example 22, although both objective functions are convex and there exists no point x∈SOC(n) such thatxT_{Qx= 0 and b}T_{x <0, the optimal values are still}

−∞ in one case and converge to a finite optimal value as variable approaches to infinity in the

other case, respectively. Given a SOCQPproblem, if the special situation applies, it is not easy to directly discern whether the optimal value is finite or unbounded below. We need to explore

further about the reason of the interesting property.

4.3

Property of Quadratic Function at Infinity

In this section, the property of a quadratic function at infinity is analyzed. For simplicity, we still study problem (4.2).

For a fixed t > 0, assume that xTQx ≥ 0 for all x ∈ St and there is no point x ∈ St such that xTQx = 0 and bTx < 0, however, there exists at least one special point x∗ ∈ St such that x∗TQx∗ = 0 and bTx∗ = 0. Let B = {x ∈ St | xTQx = 0, bTx = 0} denote the set of all the points x ∈ St simultaneously satisfying xTQx = 0 and bTx = 0. Define

Nx∗(σ) ={x ∈S_t | kx−x∗k< σ} to be theσ−neighborhood of point x∗ on the cross-section

St. Let NB(σ) = Sx∈BNx(σ) be the union set of all the σ−neighborhoods of points x ∈ B. Notice that, by the continuity of the quadratic function, for any > 0, there exist a scalar

For a point x∗ ∈ B, assume that the index of x∗ can be parititioned into three parts I,J

and {n}: I ={i |x∗_i = 0,1≤i≤n−1} (I can not be empty),J ={i|x∗_i = 06 ,1≤i≤n−1}

and |I|+|J|=n−1. Now we make up a new vector ˆx∈_Rn _{which satisfies that: (i) x} i = ˆxi for i ∈ I, where ˆxi ∈ R is a random number; (ii) ˆxi = M x∗i for i ∈ J, where M > 0 is a big positive number; (iii) ˆxn =

q

M2_x∗2 n +

P

i∈Ixˆ2i. It is easily to verify that ˆx ∈ SOC(n). Moreover, for anyσ >0, we can find a big enoughM such that the corresponding point _xnˆˆxtin the cross-sectionSt fall into NB(σ). The objective value at ˆx is

f(ˆx) = xˆTQxˆ+bTxˆ

= P

k∈I,l∈IQklxˆkxˆl+Pi∈Ibixˆi P

r∈J,t∈JQrtxˆrxˆt+Pj∈Jbjxˆj+ 2Pi∈I,j∈JQijxˆixˆj +Qnnxˆ2n+ 2

P

i∈IQnixˆnˆxi+ 2Pj∈JQnjxˆnˆxj+bnxˆn. Since f(x∗) = 0, we haveM2f(x∗) = 0.

M2f(x∗) = M2(Qnnx∗2n + P

r∈J,t∈JQrtx∗rx∗t+ 2 P

j∈JQnjx∗nx∗j) +M(bnx∗n+ P

j∈Jbjxj). Recall that, ˆxj =M x∗j forj ∈J and ˆxn=

q

M2_x∗2 n +

P

i∈Ixˆ2i. Then,f(ˆx) can be written in the following form:

f(ˆx) = P

k∈I,l∈IQklxˆkxˆl+ P

i∈Ibixˆi+ 2 P

i∈IQnixˆnxˆi+ 2 P

i∈I,j∈JQijxˆixˆj +Qnn(ˆx2_n−M2x_n∗2) +bn+1(ˆxn−M x∗n)

+2MP

j∈JQnj(ˆxnxj∗−M x∗nx∗j)

.

From the definition of ˆxn, we can get

lim

M−→∞xˆn−M x ∗

n= lim M−→∞

P i∈Ixˆ2i q

M2_x∗2 n +

P

i∈Ixˆ2i +M x∗n

= 0, (4.7)

lim

M−→∞M(ˆxn−M x ∗

n) =_M−→∞lim

P i∈Ixˆ2i q

x∗2 n +

P

i∈Ixˆ2i M2 +x∗n

= P

i∈Ixˆ2i 2x∗

n

. (4.8)

Therefore, for any > 0, there exist big values N1() and N2() such that if M ≥ N1(), 0<xˆn−M xn∗ < ; ifM ≥N2(),

P

i∈Ixˆ2i 2x∗

n − < M(ˆxn−M x ∗ n)<

P

i∈Ixˆ2i 2x∗

n . Thus whenM ≥max(N1(), N2()), we have that

f(ˆx) = P

k∈I,l∈IQklxˆkxˆl+Pi∈Ibixˆi+ 2Pi∈IQnixˆnxˆi+ 2Pi∈I,j∈JQijxˆixˆj +QnnPi∈Ixˆ2i +

P

i∈Ixˆ2i x∗

n P

j∈JQnjx∗j+O()

Furthermore, if we just focus on the quadratic formxT_{Qx, whenM ≥max(N}

ˆ

xTQxˆ = P

k∈I,l∈IQklxˆkxˆl+QnnPi∈Ixˆ2i +

P

i∈Iˆx2i x∗

n P

j∈JQnjx∗j +2P

i∈Ixˆi(Qnixˆn+Pj∈JQijxˆj) +O()

= P

k∈I,l∈IQklxˆkxˆl+QnnPi∈Ixˆ2i +

P

i∈Iˆx2i x∗

n P

j∈JQnjx∗j +2MP

i∈Ixˆi(Qnix∗n+ P

j∈JQijx∗j) +2P

i∈IxˆiQni(ˆxn−M x∗n) +O()

= P

k∈I,l∈IQklˆxkxˆl+QnnPi∈Ixˆ2i +

P

i∈Iˆx2i x∗

n P

j∈JQnjx∗j +2MP

i∈Ixˆi(Qnix∗n+ P

j∈JQijx∗j) +O()

SincexTQx≥0 for allx∈St, thusxTQx≥0 for allx∈SOC(n). Based on this fact, we prove the following theorem.

Theorem 24. Let B={x∈St |xTQx= 0, bTx= 0}. SupposexTQx≥0for all x∈St, then for any point x∗ ∈ B satisfies that the index set I ={i |x∗_i = 0,1≤i≤ n−1} 6=∅, we have that2Qnix∗n+

P

j∈JQijx∗j = 0 for any i∈I, where J ={i| x∗i 6= 0,1≤i≤n−1}. Proof. Assume that ∃ˆi ∈ I such that Q_nˆ_ix∗n +

P

j∈JQˆ_ijx∗j = p 6= 0. We can make up a new point ˆx such that ˆxˆ_i = −p, ˆxi = 0 for other i ∈ I, ˆxi = M x∗i for i ∈ J and ˆ

xn = q

M2_x∗2 n +

P

i∈Ixˆ2i. It is easy to verify that ˆx ∈ SOC(n). Let be a small enough number. When M ≥max(N1(), N2()), ˆxTQxˆ = 2

P

i∈Ixˆ2i 2x∗

n P

j∈JQnjx∗j + P

k∈I,l∈IQklxˆkxˆl+

Qnn P

i∈Ixˆ2i+2M P

i∈Ixˆi(Qnix∗n+ P

j∈JQijx∗j)+O(). Since ˆxi is finite fori∈I, we can find a constantT such that|P

k∈I,l∈IQklxˆkxˆl+QnnPi∈Ixˆ2i+

P

i∈Iˆx2i x∗

n P

j∈JQnjx∗j|+O()< T. Then, if M ≥max(N1(), N2(),_−pT2), we have that ˆxTQx <ˆ 0. This contradicts the fact xTQx ≥0

for all x∈SOC(n). Consequently, Qnix∗n+ P

j∈JQijx∗j = 0 for any i∈I.

When M ≥ (N1(), N2()), Theorem 24 implies that ˆxTQxˆ and f(ˆx) can be written as follows:

ˆ

xTQxˆ = P

i∈I(Qii+Qnn+

P

j∈JQnjx

∗

j x∗

n )ˆx 2 i +

P

k∈I,l∈I,k6=lQklxˆkxˆl+O() .

f(ˆx) = P

k∈I,l∈IQklxˆkxˆl+Qnn P

i∈Ixˆ2i +

P

i∈Ixˆ2i x∗

n P

j∈JQnjx∗j +P

i∈Ibixˆi+O()

= P

i∈I(Qii+Qnn+

P

j∈JQnjx∗j x∗

n )ˆx 2 i +P

k∈I,l∈I,k6=lQklxˆkxˆl+ P

i∈Ibixˆi+O() Define ¯Q∗_II =QII +diag(Qnn +

P

j∈JQnjx

∗

j x∗

n+1 ) ∈ S

m _{to be the corresponding matrix for x}∗_. Based on the new reformulation off(ˆx), another interesting theorem can be stated.

Theorem 25. Let B={x∈St |xTQx= 0, bTx= 0}. SupposexTQx≥0for all x∈St, then for any point x∗ ∈ B satisfies that the index set I ={i | x∗_i = 0,1 ≤i≤n−1} 6=∅, Q¯∗_II is a positive semidefinite matrix.

such that yTQ¯∗_IIy = r < 0. Let J = {1, . . . , n−1} −I. We can make up a new vector ˆ

x ∈ Rn _{such that ˆx}

ˆ_i = yi for i ∈ I, ˆxi = M x∗i for i ∈ J and ˆxn = q

M2_x∗2 n +

P i∈Ixˆ2i. It is easy to verify that ˆx ∈ SOC(n). Since ˆxi is finite for i ∈ I, we can find a constant T such that |P

k∈I,l∈I,k6=lQklxˆkxˆl|+O() < T. When M ≥ max(N1(), N2(), q

T

r), we have ˆ

xTQx <ˆ 0. This contradicts the factxTQx≥0 for all x ∈SOC(n). Therefore, ¯Q∗_II is positive semidefinite.

For a vector d ∈ _Rn _{with an index set I ⊆ {1, . . . , n}, |I| = m, let b}

I ∈ Rm denote the subvector ofbwhich takes the element bi fori∈I. We can write ˆxTQxˆand f(ˆx) as follows:

ˆ

xTQxˆT = ˆxT_IQ¯∗_IIxˆI+O(), f(ˆx) = ˆxT_IQ¯∗_IIxˆI+bT_IxˆI+O(). Based on the results above, we give the following theorem:

Theorem 26. Let B={x∈St |xTQx= 0, bTx = 0} and suppose xTQx≥0 for all x ∈St. For all pointsx∗ ∈ B satisfy that the index setI ={i|x∗_i = 0,1≤i≤n−1} 6=∅, ifbI is in the column space of Q¯∗_II, then the objective function f(x) takes a finite optimal value. Otherwise,

f(x) can be unbounded below.

Proof. For any point x∗ ∈ B, we make up a vector ˆx such that ˆxi ∈ R for i ∈ I, ˆxi = M x∗i for i ∈ J and ˆxn =

q

M2_x∗2 n +

P

i∈Ixˆ2i. It is easy to verify that ˆx ∈ SOC(n). Notice that, for a large enough number M, minimizing f(ˆx) is equivalent to minimizing ˆxT_IQ¯∗_IIxˆI+bTIxˆI for arbitrary ˆxI ∈ Rm. Thus, from the KKT condition, if for any x ∈ B with I 6= ∅, the corresponding bI is in the column space of ¯QII, f(ˆx) achieves a finite optimal value. On the other hand, if there exists one pointx∗ ∈ B withI 6=∅ such thatbI is not in the column space of ¯QII, then a feasible descending direction can always be found for any point in SOC(n) with a nonzero step length. Thus, f(ˆx) is unbounded below.

Above all, the following two different situations explain the interesting property of the second-order cone SOC(n) in the special caseB 6=∅.

1. If there exists a x∗ ∈ B with I 6=∅ such that the corresponding bI is not in the column space of ¯QII, then the original problem is unbounded below.

4.4

Properties of Conic Reformulation

This section uses the cone of nonnegative quadratic functions to get the conic reformulation. For a given setF ⊂_Rn_{, the cone of nonnegative quadratic functions [120] is defined as follows:}

DF = (

M ∈ Sn+1 _|M· "

1 xT x xxT

#

≥0,∀x∈ F

)

. (4.9)

DF is the set of the coefficients of quadratic functions which are nonnegative over the feasible domain of F. We also define another cone over F as

D∗_F = cl Cone (

X= "

1

x

# " 1

x

#T

|x∈ F

)

. (4.10)

DF and D∗F are both closed and convex. From Corollary 1 of [120], they are actually dual to each other.

Example 27. If F =Rn, D∗ F =S

n+1

+ =DF.

Moreover, the homogenized set ofF is defined as

HF =cl ("

t

x

#

∈_Rn+1_| x

t ∈ F, t >0 )

. (4.11)

Some monotonic properties of the cone of nonnegative quadratic functions and their dual cones are shown in the next lemma.

Lemma 28. If F1 ⊆ F2, then DF∗1 ⊆ D

∗

F2 and DF2 ⊆ DF1. Therefore, for any F ⊆ R

n_,

D∗

F ⊆ S+n+1 ⊆ DF.

Proof. From the definitions of the cones of nonnegative quadratic functions and their dual cones, we can directly get this result.

The next lemma provides a scheme to do rank-one decomposition. Lemma 29. Suppose X∈ Sn

+ is not zero andrank(X) =r. For any vectora∈Rn, ifXa 6= 0, thenX0 =X−XaaTX

aT_Xa ∈ S+n and rank(X0) =r−1. Proof. SinceX ∈ Sn

+andXa6= 0, thenaTXa >0. ∀u∈Rn, letX0 =X−Xaa T_X

aT_Xa . We have that

uTX0u=uTXu−(uTXa)2

aT_Xa . SupposeX=YTY and let ¯a=Y aand ¯u=Y u. Then from Cauchy-Schwarz inequality, we have (uTXu)(aTXa) = kuk¯ 2_k¯ak2 _{≥ (¯u}T_¯a)2 _{= (u}T_Xa)2_{. Therefore,}

uTX0u ≥ 0 for any u ∈ _Rn_{. Therefore, X}0 _{∈ S}n