Topics in quadratic binary optimization problems

(1)

Topics in quadratic binary optimization

problems

by

Pooja Pandey

M.Sc., University of New Brunswick, 2008 M.Sc., Indian Institute of Technology, Kanpur, 2000

B.Sc., Kanpur University, 1998

Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of

Doctor of Philosophy in the

Department of Mathematics Faculty of Science

©Pooja Pandey 2018

SIMON FRASER UNIVERSITY Summer 2018

However, in accordance with the Copyright Act of Canada, this work may be reproduced without authorization under the conditions for “Fair Dealing.” Therefore, limited reproduction of this work for the purposes of private study, research, education, satire, parody, criticism, review and news reporting is likely

(2)

Approval

Name: Pooja Pandey

Degree: Doctor of Philosophy (Operations Research)

Title: Topics in quadratic binary optimization problems

Examining Committee: Chair: J. F. Williams

Associate Professor Abraham P. Punnen Senior Supervisor Professor Ramesh Krishnamurti Supervisor Professor

School of Computing Science

Binay Bhattacharya

Internal Examiner Professor

School of Computing Science

Haibo Wang

External Examiner

Killam Distinguished Professor in Decision Sciences and Operation Research,

Division of International Business & Technology Studies,

Texas A& M International University

Date Defended: July 12 2018

(3)

Abstract

In this dissertation, we consider the quadratic combinatorial optimization problem (QCOP) and its variations: the generalized vertex cover problem (GVC), the quadratic unconstrained binary optimization problem (QUBO), and the quadratic set covering problem (QSCP). We study these problems as discussed below. For QCOP, we analyze equivalent representations of the pair (Q, c), where Q is a quadratic cost matrix and c is a linear cost vector. We present various procedures to obtain equivalent representations, and study the effect of equivalent representations on the computation time to obtain an optimal solution, on the quality of the lower bound values obtained by various lower bounding schemes, and on the quality of the heuristic solution. The first variation of QCOP is GVC, and we show that GVC is equivalent to QUBO and also equivalent to some other variations of GVC. Some solvable cases are identified and approximation algorithms are suggested for special cases. We also study GVC on bipartite graphs. QUBO is the second variation of QCOP. For QUBO, we analyze several heuristic algorithms using domination analysis. We show that for QUBO, if any algorithm that guarantees a solution no worse than the average, has a domination ratio of at least 1/40. We extend this result to the maximum and minimum cut problems; maximum and minimum uncut problems; and GVC. We also study the QUBO when Q is: 1) (2d + 1)-diagonal matrix, 2) (2d + 1)-reverse-diagonal matrix, and 3) (2d+1)-cross diagonal matrix, and show that these cases are solvable in polynomial time when d is fixed or is of O(log n). The last variation of QCOP is QSCP. For QSCP, we identify various inaccuracies in the paper by R. R. Saxena and S. R. Arora, A linearization technique for solving the Quadratic Set Covering Problem, Optimization, 39 (1997) 33-42. In particular, we observe that their algorithm does not guarantee optimality, contrary to what is claimed. We also present the mixed integer linear programming formulations (MILP) for QSCP. We compare the lower bounds obtained by the linear programming relaxations of MILPs, and study the effect of equivalent representations of QSCP on these MILPs.

Keywords: combinatorial optimization problem; equivalent representation; set covering

problem; domination analysis; 0-1 quadratic programming; generalized vertex cover problem

(4)

Dedication

To my mother and father for allowing me to pursue what I like and being my support system, my husband Vikas for always standing by me, and my daughter Poorvi for unconditional love.

(5)

Acknowledgements

I take this opportunity to thank my senior supervisor, Prof. Abraham P. Punnen, for his guidance and support throughout my Ph.D. studies. He has provided me with invaluable advice, helpful comments, and suggestions, while conducting research. When I had trouble in moving forward with some of the research problems, he has helped me by providing par-tial results. This thesis would not have been possible without his help. I am also thankful to other members of my dissertation advisory committee - Prof Ramesh Krishnamurti, Prof. Haibo Wang, and Prof. Binay Bhattacharya - for taking time to examine my work and providing constructive feedback.

Special thanks to my husband Vikas and daughter Poorvi for their unconditional love and support during these many years of graduate school. The journey has been long, and at times very stressful; however, their comforting presence made it easier. I deeply appreciate all sacrifices both of them have made to help me achieve my goals. I also thank my parents, brothers, and family members for their constant encouragement, without their support and encouragement, this study would never have been completed.

I would like to thank postdoctoral fellows and graduate students Ante, Piyashat, Brad, TJ, Krishna, Sherry, and many more in the Operation Research group, it was indeed a pleasure to work with all them.

I also extend my gratitude to the faculty and staff members of the Department of Math-ematics for their support. I would like to express my sincere appreciation to the staff of the SFU Research Commons for their support during the Thesis Boot Camp, specially to Robyn Long and Poh Tan for their invaluable advice on the academic writing. I would like to acknowledge the financial support provided by Prof Abraham P. Punnen for my research program. Finally, I would like to thank the Dean of Graduate studies SFU for awarding me Travel and Minor Research Awards and Graduate Fellowships.

(6)

List of Tables

Table 2.1 Test instance analysis . . . 30

Table 2.2 Classes of quadratic cost matrices used in the experiments . . . 31

Table 2.3 Summary of results . . . 33

Table 2.4 Q non-negative and in [5,10], M= 10000 . . . 34

Table 2.5 Q positive semidefinite and B in [-5,5], M= 10000 . . . 35

Table 2.6 Q non-negative and positive semidefinite, B in [5,15], M= 10000 . . . 36

Table 2.7 Q arbitrary(symmetric distribution) is in [-5,5], M= 10000 . . . 37

Table 2.8 Q arbitrary(left-skewed distribution) is in [-10,5], M= 10000 . . . 38

Table 2.9 Q arbitrary(right-skewed distribution) is in [-5,10], M= 10000 . . . . 39

Table 2.10 Q Rank 1, a in [-10,10] in b [-5,5], M=10000 . . . 40

Table 2.11 Q Rank 2, a1, a2 in [-10,10] in b1, b2 in [-5,5], M=10000 . . . 41

Table 2.12 Summary of results: Heuristic value . . . 42

Table 2.13 Q non-negative is in [5,10], M= 10000, Time Limit 15 minutes and 30 minutes . . . 43

Table 2.14 Q positive semidefinite [-5,5], M= 10000, Time Limit 15 minutes and 30 minutes . . . 44

Table 2.15 Q non-negative positive semidefinite [5,15], M= 10000, Time Limit 15 minutes and 30 minutes . . . 45

Table 2.16 Q arbitrary symmetric [-5,5], M= 10000, Time Limit 15 minutes and 30 minutes . . . 46

Table 2.17 Q arbitrary left-skewed [-10,5], M= 10000, Time Limit 15 minutes and 30 minutes . . . 47

Table 2.18 Q arbitrary right-skewed [-5,10], M= 10000, Time Limit 15 minutes and 30 minutes . . . 48

Table 2.19 Q Rank 1 , M= 10000, Time Limit 15 minutes and 30 minutes . . . . 49

Table 2.20 Q Rank2, M= 10000, Time Limit 15 minutes and 30 minutes . . . 50

Table 2.21 Frequency table for tighter NLB bounds (32 test instances for each class) . . . 51

Table 2.22 Natural lower bound calculation: Q non-negative in [5,10] . . . 52 Table 2.23 Natural lower bound calculation: Q positive semidefinite B in [-5,5] . 53

(10)

Table 2.24 Natural lower bound calculation: Q non-negative and positive

semidef-inite B in [5,15] . . . 54

Table 2.25 Natural lower bound calculation: Q arbitrary symmetric in [-5,5] . . 55

Table 2.26 Natural lower bound calculation: Q arbitrary left-skewed in [-10,5] . 56 Table 2.27 Natural lower bound calculation: Q arbitrary right-skewed in [-5,10] 57 Table 2.28 Natural lower bound calculation: Q Rank 1, a in [-10,10] in b in [-5,5] 58 Table 2.29 Natural lower bound calculation: Q Rank 2, a in [-10,10] in b in [-5,5] 59 Table 2.30 Summary of results for natural lower bounds for minimum time (rep-resentation) . . . 60

Table 3.1 Abbreviation of different problems . . . 91

Table 6.1 Benchmark instances, D is positive semi-definite . . . 151

Table 6.2 Benchmark instances, D is non-negative and positive semi-definite . . 152

Table 7.1 Size of Linearization . . . 164

Table 7.3 Summary of results: Best equivalent representation for MILPs . . . . 167

Table 7.4 Summary of results: Best MILP for equivalent representations . . . . 167

Table 7.5 Frequency table: Frequency of equivalent representations . . . 169

Table 7.6 Frequency table: Frequency of MILPs . . . 170

Table 7.7 Q non-negative in [5,10], Table 1 . . . 171

Table 7.8 Q non-negative in [5,10], Table 2 . . . 172

Table 7.9 Q positive semidefinite and B is in [-5,5], Table 1 . . . 173

Table 7.10 Q positive semidefinite and B is in [-5,5], Table 2 . . . 174

Table 7.11 Q non-negative positive semidefinite and B is in [5,15], Table 1 . . . 175

Table 7.12 Q non-negative positive semidefinite and B is in [5,15], Table 2 . . . 176

Table 7.13 Q arbitrary symmetric is in [-5,5], Table 1 . . . 177

Table 7.14 Q arbitrary symmetric is in [-5,5], Table 2 . . . 178

Table 7.15 Q arbitrary left-skewed is in [-10,5], Table 1 . . . 179

Table 7.16 Q arbitrary left-skewed is in [-10,5], Table 2 . . . 180

Table 7.17 Q arbitrary right-skewed is in [-5,10], Table 1 . . . 181

Table 7.18 Q arbitrary right-skewed is in [-5,10], Table 2 . . . 182

Table 7.20 Summary of computational results . . . 191

Table 7.21 Q non-negative in [5,15] . . . 192

Table 7.22 Q positive semidefinite B in [-5,5] . . . 193

Table 7.23 Q non-negative positive semidefinite B in [5,15] . . . 194

Table 7.24 Q arbitrary Symm distribution [-5,5] . . . 195

Table 7.25 Q arbitrary LS distribution [-10,5] . . . 196

(11)

Table 7.26 Q arbitrary RS distribution [-5,10] . . . 197

(12)

List of Figures

Figure 5.1 Lower d-block matrix . . . 119

Figure 5.2 (2d + 1)-diagonal matrix . . . . 121

Figure 5.3 (2d + 1)-reverse-diagonal matrix . . . . 121

Figure 5.4 (2d + 1)-cross-diagonal matrix . . . . 121

Figure 5.5 Examples of (d=2) 2-band support graph(left) and (d=3) 3-band graph(right) (n=5). . . 122

Figure 5.6 Illustration of S_lt 1 and S t l2 with Pl(t) and ¯Pl(t) . . . . 125

Figure 5.7 Examples of 1-band bipartite graph(left) and 2-band bipartite graph(right) (m = 5, n = 5). . . . 128

Figure 5.8 Examples of a reverse 2-band support graph (d = 2, and n = 10). . 129

Figure 5.9 Examples of (d=2) cross 2-band support graph for n = 10. . . . 134

Figure 5.10 Examples of (d=2) 2-band support graph(left) and (d=3) 3-band graph(right) (n=5). . . 135

(13)

Chapter 1

Introduction

A combinatorial optimization problem (COP) is to pick the best solution from a finite set of feasible solutions which can be stated as an ordered pair (f,F ), where f is the objective function and F is the finite set of feasible solutions. The complexity nature of the COP depends on the structure of the family of feasible solutionsF and the nature of the objec-tive function f . If f is a linear function, corresponding problem is the linear combinatorial optimization problem (LCOP) and if f is a quadratic function, the corresponding problem is the quadratic combinatorial optimization problem (QCOP). The COP arises in many ar-eas of Operations Research and Computer Science, some of the examples are: partitioning, packing, covering sets; portfolio selection; combinatorial auctions winner determination; protein structure prediction; Internet data packet routing; partitioning graphs or digraphs etc.

In this thesis, we consider four special cases of COP: the quadratic combinatorial op-timization problem (QCOP), the generalized vertex cover problem (GVC), the quadratic unconstrained binary optimization problem (QUBO), and the quadratic set covering prob-lem (QSCP). Moreover, QCOP is the generalization of GVC, QUBO, and QSCP. In this chapter, we provide detailed definitions of these problems, followed by the outline and the contribution of this thesis, and conclude with a list of partial publications from our thesis work.

1.1 The quadratic combinatorial optimization problem

First we present the mathematical formulations of LCOP and QCOP. Let E = {1, 2, . . . , n} be a finite set and ˆF be a family of subsets of E. For each j ∈ E, a cost cj is prescribed.

Further, for each (i, j) ∈ E × E, a cost q_ij is also prescribed. Note that any S ∈ ˆF can be represented by its 0 − 1 incidence vector x = (x1, . . . , xn) where xj = 1 if and only if j ∈ S.

Thus ˆF can be represented asF = {x ∈ {0, 1}n: x is an incidence vector of some S ∈ ˆF }.

(14)

Let Q be the n × n matrix such that its (i, j)th element is q_ij and c = (c₁, . . . , cn). Then,

the linear combinatorial optimization problem (LCOP) is to

Minimize cx Subject to x ∈F

and the quadratic combinatorial optimization problem (QCOP) is to

Minimize cx + xTQx Subject to x ∈F or equivalently Minimize X i∈S ci+ X i∈S X j∈S qij Subject to S ∈ ˆF .

The LCOP specializes into different well known optimization problems depending on the nature of f and the structure of F . Some examples from the graph theory are: If G is a balanced bipartite graph, where F is the family of perfect matchings in G and f represents the cost of a perfect matching, then the LCOP becomes the linear assignment problem (LAP). If G is an undirected graph,F is the family of all spanning trees on G, and f represents the cost of a spanning tree, then the LCOP reduces to the minimum spanning tree problem (MSTP) [46]. If G is an undirected graph,F is the family of all Hamiltonian tours of G, and f represents the cost of a Hamiltonian tour, then the LCOP is the well known traveling salesman problem (TSP). An example of LCOP which is not a graph the-oretical problem is the Knapsack problem (KP), and it can be defined as follows. Given a Knapsack of the fixed capacity and a set of n items, each item has a weight and a value. The Knapsack problem is to obtain a combination of units of the specified items such that the total value of the selected items is maximized. These special cases of LCOP are the classic combinatorial optimization problems and have been thoroughly studied in the literature with the wide range of applications. For example, the LAP has applications in the field of vehicle routing, DNA sequencing etc [37]; the MSTP has applications in the field of network design, telecommunication, Internet and transport networks [71], electrical grids, and many others [123]; the KP has applications in the field of resource allocation, capital investment selection etc [106]; and the TSP has applications in the field of computer wiring, X-Ray crystallography etc [11]. Among the mentioned problems, the TSP is strongly NP-hard and the KP is NP-hard but the MST and the LAP can be solved in polynomial time.

(15)

Like the LCOP, the QCOP also generalizes many well known combinatorial problems in-cluding the LCOP. If qij = 0 for all i, j ∈ E, then the QCOP reduces to an LCOP. Some well

known examples of QCOP are discussed here. The quadratic assignment problem (QAP) [42, 116] is a special case of QCOP and it was introduced by Koopmans and Beckmann [111]. The QAP can be defined as follows. For n facilities and n locations, let F = (fij)n×n,

D = (dkl)n×n, and B = (bik)n×n, where fij is the flow between the facility i to facility j;

dkl is the distance between location k and location l; and bik is the cost of placing facility

i to location k. Let N = {1, 2, . . . , n} and Sn is the set of all permutations φ : N → N .

Then the QAP is to select a permutation φ such that Pn i=1

Pn

i=1fijdφ(i)φ(j)+

Pn i=1biφ(i)

is minimized. In QAP, the objective function f = Pn

i=1

Pn

i=1fijdφ(i)φ(j) +Pni=1biφ(i) is

a quadratic function and Sn is the finite set of feasible solutions F ; therefore, QAP is a

special case of QCOP. The QAP is a well studied problem with many applications such as bandwidth minimization of a graph [42], image processing [164], molecular conformations in chemistry, scheduling [40, 66], backboard wiring problem in electronics [152], parallel and distributed computing [33], chip design [88], keyboard design [39], and statistical data anal-ysis [42] etc. For more applications of QAP, please refer to the survey paper [38, 42]. The QAP is perhaps one of the most intensively studied problems. The QAP is the quadratic counterpart of the LAP and LAP is polynomial time solvable. On the contrary, the QAP has been shown to be strongly NP-hard, even solving the QAP with size n = 30 by an exact algorithm is still a challenge. Moreover, for an arbitrary α > 0, there does not exist a polynomial time α-approximation algorithm for QAP unless P = NP [156].

Another special case of QCOP is the quadratic minimum spanning tree problem (QM-STP) [12, 49], which is a quadratic counterpart of MSTP. The QMSTP can be defined as follows. Let G = (V, E) be an undirected graph where |V | = n, |E| = m and F is the family of spanning trees of G. For every e ∈ E, a cost c_e is prescribed and for every e, f ∈ E, a quadratic cost qef is prescribed. Then the QMSTP is to select a T ∈ F such

that f = P

e∈Tce+Pe∈T

P

f ∈Tqef is minimized. It can be easily shown that the QMST

is a generalization of TSP, QASP, and the maximum clique problem. Some of the appli-cations of QMST are in modeling a pipeline communication problem, telecommuniappli-cations, transportation, irrigation, energy distribution, etc.

One more special case of QCOP is the quadratic Knapsack problem (QKP) which can be defined as follows. Let E = 1, 2, . . . , m be a finite set. For each j ∈ E, a positive weight w_j is prescribed and for each (i, j) ∈ E × E, a cost q_ij is prescribed, and c > 0 is a given capacity. Then the QKP is to select a subset S ⊆ E such that f =P

i∈S

P

j∈Sqij is

minimized. The QKP is a NP-hard problem as it generalizes the knapsack problem. Some of the applications of QKP are in the localization of railway stations and airports, compiler design problem etc [62, 142]. For more applications of QKP, please refer to the survey paper

(16)

[142].

Other examples of QCOP are the quadratic unconstrained binary optimization prob-lems (QUBO) [108], the quadratic travelling salesman problem [58, 130, 150], the quadratic shortest path problem [99, 153], the general quadratic 0 − 1 programming problem [29, 30, 31, 32, 61, 143, 144, 161], the quadratic set covering problem [56, 85, 133], 0-1 bilin-ear programs [50, 78], and combinatorial optimization problems with interaction costs [118].

1.2 The generalized vertex cover problem

In the previous section, we discussed QCOP and LCOP. In this section, we consider the generalized vertex cover problem, which is a special case of LCOP. In Chapter 3, we give an alternate 0-1 quadratic formulation of GVC, therefore the GVC is also a special case of QCOP. The GVC can be defined as follows. Let G = (V, E) be a graph with V = {1, 2, . . . , n} and |E| = m. For each edge (i, j) ∈ E three real valued weights q0

ij, qij1, and

q_ij2 are associated. Also, for each vertex i ∈ V a weight c_i is prescribed. For any subset U ⊆ V , let E0(U ) = {(i, j) ∈ E : i, j 6∈ U }, E1(U ) = {(i, j) ∈ E : either i ∈ U or j ∈ U but not both}, E2(U ) = {(i, j) ∈ E : i, j ∈ U }, and

f (U ) =X i∈U ci+ X (i,j)∈E0(U ) q_ij0 + X (i,j)∈E1(U ) q_ij1 + X (i,j)∈E2(U ) q2_ij.

Then the generalized vertex cover problem (GVC) is to find a set U ⊆ V such that f (U ) is minimized. Note that q0_ij can be viewed as the ‘cost of’ not covering edge (i, j), q_ij1 can be viewed as the ‘cost of’ covering edge (i, j) by selecting exactly one of its end points, and q_ij2 can be viewed as the ‘cost of’ over-covering edge (i, j). If the solution is an empty set φ, then the objective function is defined as f (φ) =P

(i,j)∈Eqij0.

GVC was introduced by Hassin and Levin [91] and it is a meaningful generalization of the well known minimum weight vertex cover problem (MWVCP) [98, 167] and the maxi-mum weight independent set problem (MWISP) [14, 167]. Note that if ci ≥ 0 for all i ∈ V,

q0

ij = M , a large number and qij1 = qij2 = 0 for all (i, j) ∈ E then, GVC reduces to MWVCP.

Likewise, when c_i ≤ 0, q2

ij = M , qij1 = qij0 = 0 for all (i, j) ∈ E, then GVC is

equiva-lent to MWISP. (It may be noted that MWISP is normally presented as a maximization problem which is equivalent to minimization form indicated above). Hassin and Levin [91] although introduced GVC, they focused primarily on a special case of it where ci is

as-sumed to be non-negative for all i ∈ V and q_ij0 ≥ q1

ij ≥ qij2 ≥ 0 for all (i, j) ∈ E. We denote

this special case of GVC by GVC-HL and it may be noted that MWISP is not a special case of GVC-HL. In [91], two 2-approximation algorithms for GVC-HL are proposed , one

(17)

based on linear programming, and the other based on the local-ratio technique [16]. When q_ij1 = α, (0 ≤ α ≤ 1), q0_ij = 1, q2_ij = 0 for all (i, j) ∈ E and ci = β, ∀i ∈ V , GVC is called

uniform cost generalized vertex cover problem (UGVC). In [91], the complexity of UGVC has been studied for all possible values of α and β. They showed that GVC is polynomial time solvable in the following cases: 1) α ≥ 1₂, 2) α < 1₂ and β ≤ 3α, 3) α < 1₂ and there exists an integer d ≥ 3 such that d(1 − α) ≤ β ≤ (d + 1)α. For the general case, UGVC is NP-hard [91]. Milanovic [127] proposed a genetic algorithm to solve GVC-HL and reported experimental results comparing their algorithm with CPLEX and the 2-approximation al-gorithm given in [91]. Kochenberger et. al. [109] compared an integer linear programming formulation and an integer quadratic programming formulations using the CPLEX solver.

Another special case of GVC where q_ij1 = q_ij2 = 0 for all (i, j) ∈ E was considered by Houchbaum [96] and Bar-Yehuda et al. [17]. This problem is also known as the general-ized vertex cover problem in the literature and for definiteness we denote this problem by GVC1. In fact, Houchbaum [96] and Bar-Yehuda et al. [17] studied primarily a special case of GVC1 where ci≥ 0 for all i ∈ V , q0ij ≥ 0 for all (i, j) ∈ E. We refer to this version of

GVC1 as GVC1-HB. Houchbaum [96] provided an integer linear programming formulation of GVC1-HB and showed that the corresponding linear programming relaxation admits half integrality property. Bar-Yehuda et al. [17] provided an extension of a well known theorem by Nemhauser and Trotter [129] for the vertex cover problem (independent set problem) to GVC1-HB, and presented a (2 − 2/d)-approximation algorithm on graphs with maximum degree of a node is bounded above by d. They also presented a polynomial time approx-imation scheme (PTAS) for GVC1-HB on planner graphs, and a (2 − log log n/2 log n)-approximation algorithm for a general graph. In the same paper they showed that GVC1-HB is NP-hard on complete graphs but solvable in polynomial time on bipartite graphs. Note that MWVCP is trivial on complete graphs.

The generalized independent set problem introduced by Hochbaum [96] is yet another special case of GVC. Here q1

ij and qij0 are assumed to be zero for all (i, j) ∈ E, and we

denote this problem by GVC2. Hochbaum and Pathria [97] studied a special case of GVC2 where ci ≤ 0 for all i ∈ V , q2ij ≥ 0 for all (i, j) ∈ E. This version of GVC2 is denoted

by GVC2-HP and the model has applications in Forest Harvesting. GVC2-HP in [97] was presented as a maximization problem with ci ≥ 0, ∀i ∈ V and q2ij ≤ 0, ∀(i, j) ∈ E. Clearly,

this is equivalent to our definition of GVC2-HP. Kochenberger et al. [107] gave a nonlinear formulation for GVC2-HP and compared the computational effectiveness of this non-linear integer programming formulation with an available linear integer programming formulation.

(18)

1.3 The quadratic unconstrained binary optimization

prob-lem

The quadratic unconstrained binary optimization problem (QUBO) studied by various authors [1, 3, 4, 57, 105, 108, 132] and is a special case of QCOP. The QUBO can be defined as follows: Let Q = (qij) be an n × n matrix and a = (a1, . . . , an) be an

n-vector. Then the quadratic unconstrained binary optimization problem (QUBO) is to find an x = (x₁, x2, . . . , xn) ∈ {0, 1}n such that g(x) = xTQx = n X i=1 aixi+ n X i=1 n X j=1 qijxixj

is minimized (or maximized). Without loss of generality q_ii is chosen as zero for i = 1, 2, . . . , n and we assume QUBO is presented as a minimization problem.

Another problem, closely related to QUBO is the bipartite quadratic unconstrained bi-nary optimization problem (BQUBO) studied recently by many authors [55, 149, 148]. Let

˜

Q = (˜qij) be an m × n matrix, ˜a = (ã1, ã2, . . . , ãm), ˜b = (˜b1, ˜b2, . . . , ˜bn). Then BQUBO is

to find an x ∈ {0, 1}m and y ∈ {0, 1}n such that

f (x, y) = xTQy + ˜˜ ax + ˜by = m X i=1 ˜ aixi+ n X j=1 ˜ bjyj+ m X i=1 n X j=1 ˜ qijxiyj

is minimized. As observed in [149, 148], QUBO can be viewed both as a generalization as well as a special case of BQUBO.

The graph theoretical interpretation of QUBO can be given as follows. For a given matrix Q, let E(Q) = {(i, j) : q_ij 6= 0, i, j = 1, . . . , n}. The subgraph G(Q) = (V, E(Q)) of the complete graph Kn with the vertex set V = {1, 2, . . . , n} is called the support graph of

Q. Then QUBO can be reformulated in terms of G(Q) i.e; QUBO is equivalent to finding U ⊆ V such that X i∈U ai+ X (i,j)∈E(Q),i,j∈U qij is minimized.

Similarly, the graph theoretical representation of BQUBO can be defined as follows. Consider an instance of BQUBO with cost matrix ¯Q of dimension m × n. The support bipartite graph of the matrix ¯Q is the bipartite graph G[ ¯Q] = (V1, V2, E[ ¯Q]), where V1 =

(19)

{1, 2, . . . , m}, V₂ = {1, 2, . . . , n} and E[ ¯Q]) = {(i, j) : i ∈ V1, j ∈ V2, ¯qij 6= 0}. The BQUBO

can be formulated as a graph theoretic optimization problem on G[ ¯Q]) = (V1, V2, E[ ¯Q])) as

Minimized φ(U₁, U2) = X i∈U1 X j∈U2 ¯ qij + X i∈U1 ¯ ai+ X j∈U2 ¯ bj subject to: U1 ⊆ V1, U2 ⊆ V2.

It may be noted that the definition of support bipartite graph is different from that of a support graph. The support graph when the underlying graph is bipartite is different from a support bipartite graph.

The QUBO has several real life applications, some of them are in the machine scheduling [3], capital budgeting and financial analysis [124], cellular radio channel assignment [44], traffic message management problem [62], computer-aided design [112], and molecular con-formation problem [139]. QUBO is equivalent to many combinatorial optimization problems such as the maximum cut problem [35, 51], minimum uncut problem [140, 141], generalized vertex cover problem [91, 127, 133] etc. Several exact solution methods using branch-and-bound framework and its variants have been studied for QUBO [19, 30, 44, 94, 120, 137, 151].

The QUBO is known to be NP-hard and identifying polynomially solvable cases of QUBO offers theoretical insight into complicated nature of the problem and helps in de-signing exact algorithms. Many polynomially solvable cases of QUBO has been studied [?, 75, 120]. Some of the polynomially solvable cases of QUBO consider restrictions Q. When all off-diagonal elements of Q are nonnegative, QUBO is known to be polynomially solvable [69]. If Q is a fixed rank positive semidefinite matrix with c = , QUBO can be solved in polynomial time [4, 57, 69, 105]. Li et al. [120] proposed a polynomial time algo-rithm to solve QUBO with tri-diagonal cost matrix and Gu et al. [75] proposed polynomial time algorithms to solve QUBO with five -diagonal and seven-diagonal cost matrices. Let the support graph G obtained from Q as stated above. QUBO is polynomially solvable if 1) G is series-parallel [132] or G is a binary tree [136]. Li et al. showed that the QUBO defined by a logic circuit can be solved in polynomial time [120].

Similarly as QUBO, the BQUBO model also has many applications. The Maximum Bi-clique Problem (MBCP) and the Maximum Weighted BiBi-clique Problem (MWBCP), which have applications in data mining, clustering and bioinformatics [43, 166], can be solved as BQUBO as follows. Consider a bipartite graph G = (V1, V2, E). A subgraph G0 = (V₁0, V₂0, E0) of G is said to be a biclique if G0 is a complete bipartite graph. Define ¯qij = 1

if (i, j) ∈ E, otherwise, ¯qij = −M where M is a large non-negative number. Select ¯a and

(20)

¯

b to be zero vectors of size m and n respectively. Then the Maximum Biclique Problem (MBCP) in G can be solved as BQUBO using ¯Q; ¯a; ¯b as defined above. If each edge (i, j) ∈ E of G has a corresponding weight r_ij, then the Maximum Weighted Biclique Prob-lem (MWBCP) is to choose a biclique in G of maximum weight. By redefining ¯qij = rij if

(i, j) ∈ E, otherwise, ¯qij = −M where M is a large positive number. Now MWBCP can be

solved as a BQUBO. Other applications of BQUBO are in correlation clustering [43, 166], bioinformatics [43, 166], approximating matrices using {−1, 1} entries [160], etc.

It is mentioned above that QUBO remains NP-hard even if Q is of rank one. In contrast, when the rank of the associated cost matrix ¯Q is fixed, BQUBO can be solved in polynomial time [149, 148]. Computing the median objective value of BQUBO is NP-hard [149, 148]. Any algorithm for BQUBO that guarantees the objective function value is no worse than the average of the objective function values of all solutions, has a domination ratio of at least 2m+n−2 [149, 148].

1.4 The quadratic set covering problem

The set covering problem is the special case of LCOP and is a well known NP complete problem and has wide range of applications. The quadratic counter part of the set covering problem is called the quadratic set covering problem (QSCP) and is a special case of QCOP. The QSCP is an important problem with many practical applications. For example, in the wireless local area planning problem (WLAN), the problem of locating access points so as to guarantee full coverage can be approximated by the QSCP [8, 7]. Another application of QSCP is in the logical analysis of data where structural information is detected using data sets [85]. In addition, QSCP also has an application in medicine; e.g. consider positive points are ill people and negative points are healthy people and a positive pattern contains no negative points and a negative pattern contains no positive points. The problem of finding a sub-collection of patterns such that every point is covered and the volume of inter-actions between positive and negative patterns is as small as possible can be formulated as QSCP [56]. Other applications of QSCP are in facility layout problems [20], line planning in public transports [41, 102] etc.

After discussing the important applications of QSCP, we present the mathematical for-mulation of QSCP. Let I = {1, 2, . . . , m} be a finite set and P = {P₁, P2, . . . , Pn} be a

collection of subsets of I. The set J = {1, 2, . . . , n} denotes the index set for the elements of P . For each element j ∈ J , a cost cj is assigned and for each element (i, j) ∈ J × J , a

cost q_ij is also assigned. We refer c = (c₁, . . . , cn) as the linear cost vector and the matrix

Q = (qij)n×n is referred to as the quadratic cost matrix.

(21)

A subset V of J is a cover of I, if ∪_j∈VPj = I. The linear set covering problem

(LSCP) is to find a cover L = {π(1), . . . , π(l)} such that Pl

i=1cπ(i) is minimized and the

quadratic set covering problem (QSCP) is to select a cover L = {π(1), . . . , π(l)} such that Pl

i=1cπ(i)+Pli=1

Pl

j=1qπ(i)π(j) is minimized.

Consider D = (d_ij)_m×n be an m × n matrix and for each i ∈ I, the vector d_i = (d_i1, di2, . . . , din) where dij =    1 if i ∈ P_j 0 otherwise. Also, consider the decision variables x₁, x2, . . . , xn where

xj =    1 if set P_j is selected 0 otherwise.

The vector of decision variables is represented by x = (x₁, . . . , xn)T and  is a column

vector of size m where all entries are equal to 1. Then the LSCP and QSCP can be formulated respectively as 0-1 integer programs

LSCP: Minimize cx Subject to Dx ≥ , x ∈ {0, 1}n, and QSCP: Minimize cx + xTQx Subject to Dx ≥ , x ∈ {0, 1}n.

The family of feasible solutions of both LSCP and QSCP is denoted by S = {x|Dx ≥ , x ∈ {0, 1}n}. The family of feasible solutions for continuous relaxations of LSCP and QSCP is represented by ¯S = {x|Dx ≥ ,  ≤ x ≤ }. The continuous relaxation of QSCP and LSCP are denoted by QSCP0 and LSCP0 respectevely.

Bazaraa and Goode [21] introduced the quadratic set covering problem (QSCP) and proposed a cutting plane algorithm to solve it. Adams [1] and Liberti [122] proposed lin-earization techniques for binary quadratic programs. Since QSCP is a binary quadratic programming problem, these linearization techniques can be used to formulate QSCP as a 0-1 integer linear program. QSCP is known to be NP-hard and polynomial time

(22)

mation algorithms are also available [56] to solve this problem.

1.5 Organization and main contributions

This thesis is divided into four parts. The first part presents the results for QCOP, the sec-ond part focuses on GVC, the third part discusses about QUBO, and the last part presents the results related to QSCP. For each part of this thesis, our contribution is presented below in the corresponding order.

1.5.1 Part I: QCOP

Chapter 2 is in the first part of the thesis where we consider QCOP. In this chapter, we consider equivalent representations of QCOP. The objective function of a QCOP can be represented by two data points, a quadratic cost matrix Q and a linear cost vector c. Different, but equivalent, representations of the pair (Q, c) for the same QCOP are well known in literature. Research papers often state that without loss of generality we assume Q is symmetric, or upper-triangular or positive semidefinite etc. These representations however have inherently different properties. Popular general purpose 0-1 integer program-ming solvers such as GUROBI and CPLEX do not suggest a preferred representation of Q and c. Our experimental analysis discloses that GUROBI prefers the upper triangular representation of the matrix Q while CPLEX prefers the symmetric representation in a statistically significant manner on our test instances. Equivalent representations, although they preserve optimality, could alter the corresponding lower bound values obtained by various lower bounding schemes. For the natural lower bound of a QCOP, symmetric rep-resentation produced tighter bounds, in general. Effects of equivalent reprep-resentations when CPLEX and GUROBI run in a heuristic mode are also explored. Further, we review var-ious equivalent representations of a QCOP from the literature that have theoretical basis to be viewed as ‘strong’ and provide new theoretical insights for generating such equivalent representations making use of constant value property and diagonalization (linearization) of QCOP instances.

1.5.2 Part II: GVC

The second part of the thesis contains Chapter 3 where we discuss GVC. In this chapter we study the GVC, which is a generalization of various well studied combinatorial optimization problems. The GVC is shown to be equivalent to the unconstrained binary quadratic pro-gramming problem and also equivalent to some other variations of the general GVC. Some

(23)

solvable cases are identified and approximation algorithms are suggested for special cases. We also study GVC on bipartite graphs and identify some polynomially solvable cases. We show that GVC on bipartite graphs is equivalent to the bipartite unconstrained 0-1 quadratic programming problem. Integer programming formulations of GVC and related problems are presented and establish half-integrality property on some variables for the corresponding linear programming relaxations. We also discuss special cases of GVC where all feasible solutions are independent sets or vertex covers. These problems are observed to be equivalent to the maximum weight independent set problem or minimum weight vertex cover problem along with some algorithmic results.

1.5.3 Part III: QUBO

The third part of the thesis contains Chapter 4 and Chapter 5 where we discuss QUBO.

In Chapter 4, we show that for the quadratic unconstrained binary optimization prob-lem (QUBO), any algorithm that guarantees a solution with the objective function value no worse than the average of the objective function value of all solutions of QUBO have a domination ratio of at least ₄₀1. We also observe that a locally optimal solution with respect to the well known neighborhoods such as 1-flip and strict 2-flip for QUBO could have the objective function value worse than the average objective function value. However, a locally optimal solution with respect to the 1-flip and the complement neighborhood is guaranteed to be no worse than the average objective function value for the homogeneous version of QUBO. Further, the greedy algorithm is observed to produce a solution with the objective function value worse than the average objective function value while the greedy expectation algorithm applied to QUBO guarantees a solution with the objective function value no worse than the average objective function value. As a byproduct, we also obtain closed form expressions for the average of the objective function values of all solutions for the maximum and minimum cut problems, maximum and minimum uncut problems, and the generalized vertex cover problem. The domination ratio of algorithms that guarantee a solution with objective function value no worse than average value of all solutions for QUBO extends to all these problems as well.

Chapter 5 focuses on the polynomially solvable special cases of QUBO. We primarily consider three different structures of Q: 1) (2d + 1)-diagonal matrix, 2) (2d + 1)-reverse-diagonal matrix, and 3) (2d + 1)-cross 1)-reverse-diagonal matrix. Each of these special cases are shown to be solvable in polynomial time when d is fixed or is of O(log n). To the best of our knowledge, the (2d + 1)-reverse-diagonal structure for QUBO is not investigated. If Q has a (2d + 1)-reverse-diagonal structure, it is possible to permute rows and columns of Q to obtain a representation with (4d + 3)-diagonal structure. Thus, for Q with the

(24)

(2d + reverse-diagonal structure, we can apply the algorithm designed for the (2d + 1)-diagonal structure for an appropriate value of d. We also develop a direct algorithm to solve QUBO with (2d + 1)-reverse-diagonal structure. Interestingly, the proposed algorithm to solve QUBO with (2d + 1)-reverse-diagonal structure is also applicable to solve QUBO with (2d + 1)-cross-diagonal structure. Finally, we show that the circular (2d + 1)-diagonal structure is a special case of (2d + 1)-cross-diagonal structure. We also investigate the rela-tionship between (2d+1)-diagonal matrix and (2d+1)-reverse-diagonal matrix. In addition, some complexity results for QUBO also have been presented.

1.5.4 Part IV: QSCP

The last part of the thesis consists of Chapter 6 and Chapter 7 and these two chapters discuss about QSCP.

In Chapter 6, we identify various inaccuracies in the paper by R. R. Saxena and S. R. Arora, A Linearization technique for solving the Quadratic Set Covering Problem, Opti-mization, 39 (1997) 33-42. In particular, we observe that their algorithm does not guarantee optimality, contrary to what is claimed. The experimental analysis has been carried out to assess the value of this algorithm as a heuristic. The results disclose that for some classes of problems the Saxena-Arora algorithm is effective in achieving good quality solutions while for some other classes of problems, its performance is poor. We also discuss similar inaccu-racies in another related paper.

In Chapter 7, we study the mixed integer linear programming formulation (MILP) for QSCP. In particular, we consider six MILPs for QSCP obtained by well know linearization techniques. The first three MILPs are obtained by using the classical linearization of the quadratic term, the McCormick envelopes to linearize the product of integer and binary variables, and variation of McCormick envelopes; next two MILPs are obtained by base-2 expansion (binarization), and base-10 expansion; and the last MILP formulation for QSCP is obtained by replacing the quadratic term with four new binary variables. We subdivide Chapter 7 into two parts as follows.

In the first part of this chapter, we study the effects of the equivalent representations of QSCP on MILPs for QSCP. For different equivalent representations of (Q, c), an MILP has different computational behavior. We perform an experimental analysis on the opti-mal behavior of MILPs with respect to different equivalent representations of QSCP. For this study, we consider six equivalent representations of QSCP obtained by simple and well know transformations. We use a general purpose 0-1 solver CPLEX for our computational experiments. The experimental study discloses that for most of the instance classes, for

(25)

different equivalent representations, the quadratic formulation outperformed all six MILPs mentioned in this study with some exceptions. Among six MILPs considered, MILPs ob-tained by classical linearization and McCormick envelopes performed way better than the MILPs obtained by binarization, base-10 expansion for most of the instance classes. For MILPs, the equivalent representations produced by symmetrization and triangularization are preferred representations for the majority of the instance classes.

In the second part of this chapter, we present a class of lower bounds for the quadratic set covering problem (QSCP). These lower bounds are obtained by the linear programming (LP) relaxations of MILPs for QSCP mentioned in the first part of this chapter. Our main focus is to compare the lower bounds of QSCP obtained by solving the LP relaxations of these MILPs. We perform extensive computational experiments to support our theoretical results on several types of test instances. We use a general purpose linear programming solver CPLEX for our computational experiments.

This thesis is concluded with a summary of our results with further promising research directions regarding the approaches considered in this thesis.

1.6 Partial publications

Given below is a partial list of our publications from the thesis work.

1. The results from Chapter 2 on equivalent representations of QCOP have been sub-mitted for the publication. The pre-print of this paper is available in arXiv.

A. P. Punnen and P. Pandey, Representations of quadratic combinatorial optimization problems: A case study using the quadratic set covering problem, arXiv, arXiv:1802.00897v2, 2018.

2. The results on the generalized vertex cover problem from Chapter 3 have been pub-lished. in .

P. Pandey and A. P. Punnen, The generalized vertex cover problem and some varia-tions, Discrete Optimization, 2018.

3. The results on the QSCP from Chapter 6 have already been published in [133].

P. Pandey and A. P. Punnen. On a linearization technique for solving quadratic set covering problem and variations. Optimization Letters, 11:1357 –1370, 2017.

(26)

Chapter 2

Quadratic combinatorial

optimization problems

Recall that the QCOP can be defined as follows: Let E = {1, 2, . . . , n} be a finite set and ˆF be a family of subsets of E. For each j ∈ E, a cost cj is prescribed. Further, for each

(i, j) ∈ E × E, a cost q_ij is also prescribed. Note that any S ∈ ˆF can be represented by its 0 − 1 incidence vector x = (x1, . . . , xn) where xj = 1 if and only if j ∈ S. Thus ˆF can be

represented asF = {x ∈ {0, 1}n: x is an incidence vector of some S ∈ ˆF } . Let Q be the n × n matrix such that its (i, j)th element is q_ij and c = (c₁, . . . , cn). Then, the quadratic

combinatorial optimization problem (QCOP) is to Minimize cx + xTQx Subject to x ∈F or equivalently Minimize X i∈S ci+ X i∈S X j∈S qij Subject to S ∈ ˆF .

When the elements of F are represented by a collection of linear constraints in bi-nary variables, QCOP can be solved using general purpose bibi-nary quadratic programming solvers such as CPLEX [101] or GUROBI [79]. The matrix Q associated with a QCOP can be represented in many different but equivalent forms using appropriate transformations on Q and c. For example, it is possible to force Q to have properties such as Q is positive semidefinite [87], negative semidefinite [87], symmetric with diagonal entries zero [87], upper (lower) triangular [61] etc., so that the resulting problem is equivalent to the given QCOP. Many authors use one of these equivalent representations to define a QCOP. This raises the

(27)

question: “Which representation of Q is better from a computational point of view?” The answer to this question depends on how one defines a ‘better representation’. Extending the work of Hammer and Rubin [87], Billionnet et al [30] used diagonal perturbations in an ‘optimal’ way to create strong reformulations of QUBO. Billionnet [29], Billionnet et al [31, 32], and Pörn et al. [144] extended this further to include perturbations involving non-diagonal elements by making use of linear equality constraints, if any, associated with a QCOP. These reformulations force Q to be symmetric and positive semidefinite yielding strong continuous relaxation. Galli and Letchford [61] obtained strong reformulations using quadratic constraints of equality type. Although all these representations are very inter-esting in terms of obtaining strong lower bounds at the root node of a branch-and-bound search tree, they require additional computational effort that are not readily available within general purpose solvers such as CPLEX or GUROBI. To the best of our knowledge, neither CPLEX nor GUROBI makes a recommendation regarding a simple and specific represen-tation of the Q matrix that is normally more effective for their respective solver.

It is not difficult to construct examples where one representation works well for CPLEX while the same representation do not work well for GUROBI and vice versa. For exam-ple, GUROBI solved a quadratic set covering instance involving 511 constraints and 210 variables in 4933 milliseconds on a PC with windows 7 operating system, Intel 4790 i7 3.60 GHz processor and 32 GB RAM. The same problem, when represented in an equivalent form with symmetry forced, GUROBI could not solve in 3 hours. CPLEX solved the problem in 23674 milliseconds and for an equivalent representation with symmetry forced, it solved in 21588 milliseconds. For another class of problems, GUROBI solved random non-diagonal reformulations efficiently, while structured equivalent formulations where Q having proper-ties such as symmetry, triangularity, positive semidefiniteness or negative semidefiniteness, could not solve many of the problems in this class (see Table 2.5 and Table 2.6). CPLEX however solved all these reformulations, although the time taken was larger than that of GUROBI for random perturbations.

We also could not find anything in the literature regarding a preferred representation of Q for solving QCOP established through systematic experimental analysis. Motivated by this, we investigate on the representation of the Q matrix for a QCOP. Unlike the way interesting theoretical works reported in [29, 30, 31, 32, 61, 144], we are not attempting to develop optimal representation based on some desirability criteria. Our experimental re-sults in Table 2.5 and Table 2.6 substantiate the merit of investigating this line of reasoning as well. Consequently, we present various transformations that provide equivalent repre-sentations of the problem, not necessarily ‘optimal’ ones. From these reprerepre-sentations, we identify six simple and well known classes that are compared using CPLEX and GUROBI. The experimental study discloses that CPLEX prefers symmetric or symmetric with a

(28)

agonal perturbation which yields a positive definite matrix, whereas GUROBI prefers an upper triangular Q matrix. Although there are outliers, statistical significance of these ob-servations are established through Wilcoxin test [169]. We also propose ways to construct strong reformulations making use of constant value property [47, 48] associated with linear combinatorial optimization problems and the concept of diagonalizable (linearizable) cost matrices associated with a QCOP [49, 99, 104, 150].

Equivalent representation of the data could also influence lower bound calculations for a QCOP. To demonstrate its impact, we used a generalization of the well know Gilmore-Lawler lower bound [67, 116] and its variations [12]. Our experiments show that for most of the test problems we used, the strongest lower bound was obtained when used an equivalent representation where Q is forced to be symmetric, except for one class of test problems for which the upper triangular structure produced tighter bound.

To conduct experimental analysis, we selected the quadratic set covering problem (QSCP). The QSCP model have applications in the wireless local area planning and the problem of locating access points to guarantee full coverage [8]. Other application areas of QSCP include logical analysis of data [85], medicine [36, 56], facility layout problems [20], line planning in public transports [41, 102] etc. Another motivation for selecting the QSCP as our test case is that relatively fewer computational studies available for this model. Thus, this work also contributes to experimental analysis of exact and heuristic algorithms for the QSCP.

The chapter is organized as follows. In section 2.1, we discuss various equivalent repre-sentations of the QCOP. Some of these reprerepre-sentations are generated using diagonalizable (linearizable) quadratic cost matrices and linear cost vectors satisfying constant value prop-erty. Characterization of diagonalizable cost matrices for the general QCOP and for a restricted version where feasible solutions have the same cardinality are also given. We also present a natural lower bound for QCOP, that is valid under the equivalent trans-formations. Section 2.2 discusses details of the experimental platform, generation of test data, and experimental results on QSCP using CPLEX12.5 and GUROBI6.0.5 comparing selected equivalent representations for exact and heuristic solutions. Experimental analysis using the natural lower bound is also given in this section.

Throughout the chapter, we use the following notations. For a given F , a QCOP can be represented by (Q, c). The matrix Q is called the quadratic cost matrix and the vec-tor c is called the linear cost vecvec-tor. For an instance (Q, c) of a QCOP and an x ∈ F , f (Q, c, x) = xTQx + cx. FR is the continuous relaxation of F . i.e. FR is obtained by

replacing the constraints xj ∈ {0, 1} in the definition of F by 0 ≤ xj ≤ 1. For x ∈ FR

(29)

we denote f_R(Q, c, x) = xTQx + cx. For any matrix Q, Diag(Q) is the diagonal ma-trix of same size as Q with its (i, i)th element is qii and Diag(Q) represents the vector

(q₁₁, q22, . . . , qnn) . For any vector a = (a1, a2, . . . , an), Diag(a) is the n × n diagonal

ma-trix with its (i, i)th element is a_i, i = 1, 2, . . . , n. All matrices are represented using capital letters and all elements of the matrix are represented by corresponding double subscripted lower-case letters where the subscripts denoting row and column indices. Vectors in Rnare represented by bold lower-case letters. The ithcomponent of vector a is a_i, of vector ¯b, is ¯bi

etc. The transpose of a matrix Q is represented by QT. The vector space of all real valued n × n matrices with standard matrix addition and scalar multiplication is denoted by Mn.

2.1 Equivalent representations

Let (Q, c) be an instance of a QCOP. Then, (Q1, c1) is an equivalent representation of (Q, c) if f (Q, c, x) = f (Q1, c1_{, x) for all x ∈}_{F . The following remark is well known.} Remark 2.1.1: (QT, c) is an equivalent representation of (Q, c).

Theorem 2.1.2: If (Q1, c1), (Q2, c2), . . . , (Qk, ck) are equivalent representations of an in-stance (Q, c) of a QCOP then

1 Pk i=1αi h Pk i=1αiQi i ,Pk1 i=1αi h Pk i=1αici i is also an equiv-alent representation of (Q, c) wheneverPk

i=1αi 6= 0. Proof. Let A = Pk1 i=1αi h Pk i=1αiQi i and b =Pk1 i=1αi h Pk i=1αici i . Then f (A, b, x) = xTAx + bx = _P_k1 i=1αi k X i=1 xTαiQix + αicix = 1 Pk i=1αi k X i=1 αif (Qi, ci, x) = 1 Pk i=1αi k X i=1 (αif (Q, c, x)) = f (Q, c, x).

From Remark 2.1.1 and Theorem 2.1.2 we have the following well-known corollary.

Corollary 2.1.3:1₂hQ + QTi, cis an equivalent representation of (Q, c).

We call the equivalent representation 1₂hQ + QTi, c given in Corollary 2.1.3 the symmetrization. This representation is well known and used extensively in literature. Since 1₂hQ + QTi _{is a symmetric matrix, it is sometimes viewed as a desirable}

represen-tation. However, symmetrization could also result in a matrix with increased or decreased rank. Thus the equivalent representation obtained by symmetrization could have prop-erties different from those of the original representation and this could impact the com-putational performance of different algorithms. Note that f_R(Q, c, x) = f_R(QT, c, x) =

(30)

fR(1₂

h

Q + QTi, c, x) for all x ∈FR. Thus, the symmetrization operation also preserves

the objective function value of the continuous relation of a QCOP. This property no longer holds for some other equivalent representations discussed later.

If one or more elements of Q, say qij is perturbed by ij and adjusting this by

sub-tracting ij from qji, we immediately get an equivalent representation of Q. Equivalent

representations obtained this way have the structure of Q0 discussed in the theorem below.

Theorem 2.1.4: If Y is a skew-symmetric matrix, D is a diagonal matrix, Q0= Q + Y + D, and c0 = c − diag(D), then (Q0, c0) is an equivalent representation of (Q, c).

Proof. Since xTY x = 0 and x2_i = xi for all i = 1, 2, . . . , n it follows that f (Q, c, x) =

f (Q0, c0, x).

In the above theorem, if Q is symmetric and if we want Q0 also to be symmetric, then Y must be the zero matrix. In this case, we can choose D = λI for sufficiently large λ to make Q0 a symmetric positive semidefinite matrix and hence f_R(Q0, c0, x) becomes convex. Hammer and Ruben [87] suggested using λ as the negative of the smallest eigenvalue of Q. Billionnet et al [30] proposed an ‘optimal’ choice of the matrix D in the case of quadratic unconstrained binary optimization (QUBO) problems. Their selection of D is ‘optimal’ in the sense that the resulting optimal objective function value fR(Q0, c0, x) of the

continu-ous relaxation is as large as possible yielding tight lower bounds. This method extends to QCOP with appropriate restriction on the representation ofF .

A quadratic cost matrix Q associated with a QCOP is said to be diagonalizable with respect toF if there exists a diagonal matrix D such that xTQx = xTDx for all x ∈F . The matrix D is called a diagonalization of Q with respect toF . Here after the terminology “diagonalizable" (“diagonalization") means diagonalizable (diagonalization) with respect to the underlying familiesF . Recall that x2_i = xifor all x ∈F and hence xTDx = diag(D)x,

where diag(D) is a vector of size n with its ithelement as the ithdiagonal entry of D. Diag-onalizable matrices form a subspace of the vector space Mnof all n × n real valued matrices. The concept of diagonalization indicated here is closely related to the linearization of some quadratic combinatorial optimization problems discussed in [49, 99, 104, 150] and for the case of binary variables, these two notions are the same. Since the terminology “lineariza-tion" is also used in another context in the case of QCOP [2, 161], we preferred to use the more natural and intuitive terminology diagonalization. Note that if Q is diagonalizable then QT and 1₂Q + QTare also diagonalizable. Also, any skew-symmetric matrix is diag-onalizable and a zero matrix of the same dimension is diagonalization of a skew symmetric matrix.

(31)

Theorem 2.1.5: (Q + A, c − diag(D)) is an equivalent representation of the QCOP instance (Q, c), where A is any diagonalizable matrix associated with the QCOP and D is a diago-nalization of A.

Proof. Since A is diagonalizable, xTAx = diag(D)x for all x ∈F . Thus, f (Q + A, c − diag(D), x) = xT(Q + A)x + (c − diag(D))x

= xTQx + xTAx + cx − diag(D)x = xTQx + diag(D)x + cx − diag(D)x = xTQx + cx = f (Q, c, x).

Corollary 2.1.6: If A1, A2, . . . , Am are diagonalizable matrices associated with a QCOP and α1, α2, . . . , αm are scalars, then (Q +Pmi=1αiAi, c −Pmi=1αidiag(Di)) is an equivalent

representation of the QCOP instance (Q, c) where Di is a diagonalization of Ai for i = 1, 2, . . . , m.

We can strengthen the equivalent representation given in Corollary 2.1.6 using a re-sult by Galli and Letchford [61]. Since A1, A2, . . . , Am are diagonalizable with respective diagonalizations D1, D2, . . . , Dm, our QCOP satisfies the constraints

xTAix − diag(Di)x = 0 for i = 1, 2, . . . , m, (2.1)

Since Ai is diagonalizable with diagonalization Di, 1₂Ai+ (Ai)T is a symmetric diago-nalizable matrix with Di as its diagonalization. Thus we can assume that Ai in equation (2.1) is symmetric for all i. Thus, we can apply the quadratic convex reformulation (QCR) technique discussed in [61], which designs an equivalent quadratic program with a convex objective function, to yield a strong reformulation making the resulting equivalent formu-lation have a continuous relaxation which is convex. Note that symmetric diagonalizable matrices form a subspace of the vector space Mn. We can use A1, A2, . . . , Am discussed above as a basis of this subspace and applying QCR reformulation [61] to yield stronger equivalent representations. A recent related work is by Hu and Sotirov [100], that used di-agonability to obtain strong lower bound for the quadratic shortest path problem on acyclic digraphs.

To generate equivalent representations of a QCOP using Theorem 2.1.5, Corollary 2.1.6 or by the QCR method [61] as discussed above, we need to identify associated diagonaliz-able matrices. Characterization of diagonalizdiagonaliz-able quadratic cost matrices associated with

(32)

a QCOP has been studied by different authors for specific cases, exploiting the underly-ing structure of F . This include quadratic assignment problems [104], special quadratic shortest path problems [99], the quadratic spanning tree problem [49], and the quadratic traveling salesman problem [150]. However, for the general QCOP without restricting the structure of F , the characterization of diagonalizable quadratic cost matrices do not yield rich classes like what was indicated for the special problems mentioned above. This is be-cause QUBO is a special case of QCOP where any subset of E is feasible.

Theorem 2.1.7: A quadratic cost matrix Q associated with a QCOP is diagonalizable if and only if Q = Y + U where Y is a skew-symmetric matrix and U is a diagonal matrix.

Proof. Since Y is skew-symmetric, xTY x = 0 for any x ∈ F and hence Q = Y + U is diagonalizable. Further, the diagonalization of such a Q is diag(U ). To prove the converse, it is enough to show that for the quadratic unconstrained binary optimization problem (QUBO), if Q is diagonalizable, then Q must be of the form Y + U . Note that the family of feasible solutions for QUBO is {0, 1}n. First, we prove that for QUBO, if a quadratic cost matrix Q0 is symmetric with diagonal entries zero is diagonalizable, then Q0 must be the zero matrix. Suppose that is not true. Let the (i, j)th element q_ij0 6= 0. Then by symmetry q_ji0 = q_ij0 6= 0. Now consider the solution

xk=

  

1 for k = ` for some ` 0 otherwise.

Let D be a diagonalization of Q0. Then xTQ0x = xTDx which implies d`` = 0. Since ` is

arbitrary, D must be a zero matrix. Now consider the solution

xk=    1 for k = i, j 0 otherwise.

Then xTQ0x = 2q_ij0 6= 0. Since Q0 is a diagonalizable, xTQ0x = xTDx = 0 which implies q_ij0 = 0, a contradiction. Thus for any symmetric cost matrix Q0 with diagonal entries zero of a QCOP, if Q0 is diagonalizable then Q0 must be zero. Now take any cost matrix Q of a QCOP that is diagonalizable. Let ¯Q = Q − diag(Q). Then ¯Q is diagonalizable and hence ˆQ = 1₂Q + ¯¯ QT is diagonalizable. But ˆQ is symmetric with diagonal entries zero. Then ˆQ must be a zero matrix and hence ¯Q = − ¯QT. Thus ¯Q is skew symmetric. But Q = ¯Q + diag(Q) and the result follows.

Note that Theorem 2.1.4 follows on a corollary of Theorems 2.1.5 and 2.1.7.

As observed earlier, imposing additional restrictions on the family of feasible solutions, more interesting characterizations for diagonalizability can be obtained [49, 99, 104, 150].

(33)

Let us now add a simple restriction that all elements of the underlying ˆF have the same cardinality. The resulting QCOP is called the cardinality constrained quadratic combinato-rial optimization problem (QCOP-CC).

A matrix P is said to be a weak-sum matrix [47] if there exists vectors a, b ∈ Rn such that p_ij = a_i+ b_j for i, j = 1, 2, . . . , n, i 6= j. Here a and b are called the generator vectors of P . Note that the sum of a weak-sum matrix and a diagonal matrix is a weak-sum matrix. For QCOP-CC we have the following characterization for diagonalizability.

Theorem 2.1.8: A quadratic cost matrix Q associated with a QCOP-CC is diagonalizable if and only if Q = P + Y where P is a weak-sum matrix and Y is a skew-symmetric matrix.

Proof. Let K be the cardinality of elements in the underlying ˆF defining the QCOP-CC instances. Suppose Q = P + Y where P is a weak-sum matrix and Y is a skew-symmetric matrix. Then xTQx = xTP x + xTY x = n X i=1 n X j=1 (a_i+ b_j) x_ixj− n X i=1 (a_i+ b_i) x_i+ n X i=1 piixi = Kax + Kbx − ax − bx + diag(P )x = [(K − 1)(a + b) + diag(P )] x = xTDx,

where D is a diagonal matrix with d_ii = (K − 1)(a_i+ b_i) + p_ii, i = 1, 2, . . . , n. Thus Q is diagonalizable.

Conversely, suppose Q is diagonalizable. We will show that Q is of the required form given in the theorem. To establish this necessary condition, it is enough to establish it for a special case of QCOP(K). So, consider the quadratic minimum spanning tree problem (QMST) on a complete graph. Custic and Punnen [49] showed that a symmetric quadratic cost matrix associated with a QMST is diagonalizable if and only if it is a weak-sum ma-trix. Consider a quadratic cost matrix Q for the QMST. Now, Q is diagonalizable if and only if 1₂Q + QT is diagonalizable. Since 1₂Q + QTis symmetric, it follows from [49] that 1₂Q + QT is a weak-sum matrix. But Q = 1₂Q + QT+ 1₂Q − QT. Since

1 2

Q − QTis skew-symmetric, the result follows.

Corollary 2.1.9: Let Y be a skew-symmetric matrix, U be a diagonal matrix, and P be a weak-sum matrix with generator vectors a and b. If Q0 = Q + Y + U + P , and c0 = c − diag(U ) − (K − 1)(a + b) − diag(P ) then (Q0, c0) is an equivalent representation of (Q, c) for QCOP-CC, where K is the fixed cardinality of elements of ˆF defining the QCOP-CC instances.

Topics in quadratic binary optimization problems