Alexander Schrijver

(1)

(2)

THEORY OF LINEAR

AND

INTEGER PROGRAMMING

(3)

WILEY-INTERSCIENCE

SERIES IN DISCRETE MATHEMATICS AND OPTIMIZATION ADVISORY EDITORS

RONALD L. GRAHAM

AT & T Bell Laboratories, Murray Hill, New Jersey, U.S.A.

JAN KAREL LENSTRA

Department of Mathematics and Computer Science, Eindhoven University of Technology, Eindhoven. The Netherlands

ROBERT E. TARJAN

Princeton University, New Jersey, and NEC Research Institute, Princeton, New Jersey, U.S.A.

AARTS AND KORST Simulated Annealing and Boltzmann Machines: A Stochastic Approach to Combinatorial Optimization and Neural Computing

AARTS AND LENSTRA Local Search in Combinatorial Optimization ALON, SPENCER, AND ERDOS The Pobabilistic Method

ANDERSON AND NASH Linear Programming in Infinite-Dimensional Spaces: Theory and Application

AZENCOTT Simulated Annealing: Parallelization Techniques

BARTHELEMY AND GUENOCHE Trees and Proximity Representations BAZARRA, JARVIS, AND SHERALI Linear Programming and Network Flows CHONG AND ZAK An Introduction to Optimization

COFFMAN AND LUEKER Probabilistic Analysis of Packing and Partitioning Algorithms COOK, CUNNINGHAM, PULLEYBLANK, AND SCHRIJVER Combinatorial Optimization DASKIN Network and Discrete Location: Modes, Algorithms and Applications

DINITZ AND STINSON Contemporary Design Theory: A Collection of Surveys ERICKSON Introduction to Combinatorics

GLOVER, KLINGHAM, AND PHILLIPS Network Models in Optimization and Their Practical Problems

GOLSHTEIN AND TRETYAKOV Modified Lagrangians and Monotone Maps in Optimization GONRAN AND MINOUX Graphs and Algorithms (Translated by S. Vajdi)

GRAHAM, ROTHSCHILD, AND SPENCER Ramsey Theory, Second Edition GROSS AND TUCKER Topological Graph Theory

HALL Combinatorial Theory, Second Edition JENSEN AND TOFT Graph Coloring Problems

LAWLER, LENSTRA, RINNOOY KAN, AND SHMOYS, Editors The Traveling Salesman Problem: A Guided Tour of Combinatorial Optimization

LEVITIN Perturbation Theory in Mathematical Programming Applications MAHMOULD Evolution of Random Search Trees

MARTELLO AND TOTH Knapsack Problems: Algorithms and Computer Implementations McALOON AND TRETKOFF Optimization and Computational Logic

MINC Nonnegative Matrices

MINOUX Mathematical Programming: Theory and Algorithms (Translated by S. Vajdi)

MIRCHANDANI AND FRANCIS, Editors Discrete Location Theory NEMHAUSER AND WOLSEY Integer and Combinatorial Optimization

NEMIROVSKY AND YUDIN Problem Complexity and Method Efficiency in Optimization

(Translated by E. R. Dawson)

PACH AND AGARWAL Combinatorial Geometry

PLESS Introduction to the Theory of Error-Correcting Codes, Second Edition

ROOS, TERLAKY, AND VIAL Theory and Algorithms for Linear Optimization: An Interior Point Approach

SCHEINERMAN AND ULLMAN Fractional Graph Theory: A Rational Approach to the Theory of Graphs

SCHRIJVER Theory of Linear and Integer Programming

TOMESCU Problems in Combinatorics and Graph Theory (Translated by R. A . Melter) TUCKER Applied Combinatorics, Second Edition

(4)

THEORY

OF

LINEAR AND

INTEGER PROGRAMMING

Centrum voor Wiskunde en Informatica, Amsterdam

A

Wiley-Interscience Publication

JOHN WILEY

&

SONS

(5)

A. Schrijver, CWI, Kruislaan 413, 1098 SJ Amsterdam, The Netherlands

Copyright F,1986 by John Wiley & Sons Ltd, Baffins Lane, Chichester, West Sussex PO19 IUD, England

National 0 1243 779777 International (+44) 1243 779777

e-mail (for orders and customer service enquiries): [email protected] Visit our Home Page on http://www.wiley.co.uk

or

http://www.wiley .com First published as paperback by John Wiley, 1998 Reprinted January 1999

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except under the terms of the Copyright, Designs and Patents Act 1988 or under the terms of a licence issued by the Copyright Licensing Agency, 90 Tottenham Court Road, London, UK W l P 9HE, without the permission in writing of the publisher.

Other Wiley Editorial Ofices

John Wiley & Sons, Inc., 605 Third Avenue, New York, NY 10158-0012, USA

Wiley-VCH Verlag GmbH, Pappelallee 3, D-69469 Weinheim, Germany

Jacaranda Wiley Ltd, 33 Park Road, Milton, Queensland 4064, Australia

John Wiley & Sons (Asia) Pte Ltd, 2 Clementi Loop #02-01, Jin Xing Distripark, Singapore 05 12

John Wiley & Sons (Canada) Ltd, 22 Worcester Road, Rexdale, Ontario M9W ILI, Canada

British Library Cataloguing in Publication Data

A catalogue record for this book is available from the British Library ISBN 0 471 98232 6

Printed and bound in Great Britain by Bookcraft (Bath) Ltd

This book is printed on acid-free paper responsibly manufactured from sustainable forestry, in which at least two trees are planted for each one used for paper production.

(6)

Preface

There exist several excellent books on linear and integer programming. Yet, I

did not feel it superfluous to write the present book. Most of the existing books focus on the, very important, algorithmic side of linear and integer programming. The emphasis of this book is on the more theoretical aspects, and it aims at complementing the more practically oriented books.

Another reason for writing this book is that during the last few years several interesting new results have been obtained, which are not yet all covered by books: Lovasz’s basis reduction met hod, K hachiyan’s ellipsoid method and Karmarkar’s method for linear programming, Borgwardt’s analysis of the average speed of the simplex method, Tardos’ and Megiddo’s algorithms for linear programming, Lenstra’s algorithm for integer linear programming, Seymour’s decomposition theorem for totally unimodular matrices, and the theory of total dual integrality.

Although the emphasis is on theory, this book does not exclude algorithms. This is motivated not only by the practical importance of algorithms, but also by the fact that the complexity analysis of problems and algorithms has become more and more a theoretical topic as well. In particular, the study of polynomial- time solvability has led to interesting theory. Often the polynomial-time solvability of a certain problem can be shown theoretically (e.g. with the ellipsoid method); such a proof next serves as a motivation for the design of a method which is efficient in practice. Therefore we have included a survey of methods known for linear and integer programming, together with a brief analysis of their running time. Our descriptions are meant for a quick understanding of the method, and might be, in many cases, less appropriate for a direct implementation.

The book also arose as a prerequisite for the forthcoming book Polyhedral

Combinutorics, dealing with polyhedral (i.e. linear programming) methods in

combinatorial optimization. Dantzig, Edmonds, Ford, Fulkerson, and Hoffman have pioneered the application of polyhedral methods to combinatorial optimization, and now combinatorial optimization is dissolubly connected to (integer) linear programming. The book Polyhedral Combinutorics describes these connec- tions, which heavily lean on results and methods discussed in the present book.

For a better understanding, and to make this book self-contained, we have

illustrated some of the results by combinatorial applications.

(7)

vi Preface

Several friends and colleagues have helped and inspired me in preparing this book. It was Cor Baayen who stimulated me to study discrete mathematics, especially combinatorial optimization, and who advanced the idea of compiling a monograph on polyhedral methods in combinatorial optimization. During leaves of absence spent in Oxford and Szeged (Hungary) I enjoyed the hospitality of Paul Seymour and Laci Lovasz. Their explanations and insights have helped me considerably in understanding polyhedral combinatorics and integer linear programming. Concurrently with the present book, I was involved with Martin Grotschel and Laci Lovasz in writing the book T h e Ellipsoid Method und

Comhinatoriul Optimization (Springer-Verlag, Heidelberg). Although the plans

of the two books are distinct, there is some overlap, which has led to a certain cross-fertilization. I owe much to the pleasant cooperation with my two co- authors. Also Bob Bixby, Bill Cook, Bill Cunningham, Jack Edmonds, Werner Fenchel, Bert Gerards, Alan Hoffman, Antoon Kolen, Jaap Ponstein, Andras Sebo, Eva Tardos, Klaus Trumper and Laurence Wolsey have helped me by pointing out to me information and ideas relevant to the book, or by reading and criticizing parts of the manuscript. The assistance of the staff of the library of the Mathematical Centre, in particular of Carin Klompen, was important in collecting many ancient articles indispensable for composing the historical surveys.

Thanks are due to all of them. I also acknowledge hospitality and/or financial support given by the following institutions and organizations: the Mathematical Centre/Centrum voor Wiskunde en Informatica, the Netherlands organization for the advancement of pure research Z.W.O., the University of Technology Eindhoven, the Bolyai Institute of the Attila Jozsef University in Szeged, the University of Amsterdam, Tilburg University, and the Institut fur Okonometrie und Operations Research of the University of Bonn.

Finally, I am indebted to all involved in the production of this book. I t has

been a pleasure to work with Ian McIntosh and his colleagues of John Wiley &

Sons Limited. In checking the galley proofs, The0 Beekman, Jeroen van den Berg, Bert Gerards, Stan van Hoesel, Cor Hurkens, Hans Kremers, Fred Nieuwland, Henk Oosterhout, Joke Sterringa, Marno Verbeek, Hein van den Wildenberg, and Chris Wildhagen were of great assistance, and they certainly cannot be blamed for any surviving errors.

(8)

PART I: LINEAR ALGEBRA

25

3 Linear algebra and complexity 27

3.1 Some theory, 27

3.2 Sizes and good characterizations, 29 3.3 The Gaussian elimination method, 31 3.4 Iterative methods, 36

Notes on linear algebra ₃₈

Historical notes, 38

Further notes on linear algebra, 40

PART 11: LATTICES A N D LINEAR D I O P H A N T I N E

EQUATIONS

43

4 Theory of lattices and linear diophantine equations 45

4.1 The Hermite normal form, 45

4.2 Uniqueness of the Hermite normal form, 48 4.3 Unimodular matrices, 48

4.4 Further remarks, 50

(9)

viii Contents

5 Algorithms for linear diophantine equations 52

5.1 The Euclidean algorithm, 52 5.2 Sizes and good characterizations, 54

5.3 Polynomial algorithms for Hermite normal forms and systems of linear diophantine equations, 56

6 Diophantine approximation and basis reduction 60

6.1 The continued fraction method, 60 6.2 Basis reduction in lattices, 67

6.3 Applications of the basis reduction method, 71

Notes on lattices and linear diophantine equations 76

Further notes on lattices and linear diophantine equations, 82

PART 111: POLYHEDRA, LINEAR INEQUALITIES,

AND LINEAR PROGRAMMING

83

7 Fundamental concepts and results on polyhedra, linear inequalities,

and linear programming 85

7.1 The Fundamental theorem of linear inequalities, 85 7.2 Cones, polyhedra. and polytopes, 87

7.3 Farkas’ lemma and variants, 89 7.4 Linear programming, 90 7.5 LP-duality geometrically, 92 7.6 Afine form of Farkas’ lemma, 93 7.7 Caratheodory’s theorem, 94 7.8 Strict inequalities, 94 7.9 Complementary slackness, 95 7.10 Application: max-flow min-cut, 96

8 The structure of polyhedra 99

8.1 Implicit equalities and redundant constraints, 99

8.2 Characteristic cone, lineality space, afine hull, dimension, 100 8.3 Faces, 101

8.4 Facets, 101

8.5 Minimal faces and vertices, 104 8.6 The face-lattice, 104

8.7 Edges and extremal rays, 105 8.8 Extremal rays of cones, 105 8.9 Decomposition of polyhedra, 106

8.10 Application: doubly stochastic matrices, 107 8.11 Application: the matching polytope, 109

9 Polarity, and blocking and anti-blocking polyhedra ₁₁₂

9.1 Polarity, 112

9.2 Blocking polyhedra, 1 13 9.3 Anti-blocking polyhedra, 116

(10)

Contents ix 10 Sizes and the theoretical complexity of linear inequalities

and linear programming 120

10.1 Sizes and good characterizations, 120 10.2 Vertex and facet complexity, 121

10.3 Polynomial equivalence of linear inequalities and linear programming, 124 10.4 Sensitivity analysis, 125

11 The simplex method 129

11.1 The simplex method, 129

11.2 The simplex method in tableau form, 132 11.3 Pivot selection, cycling, and complexity, 137 11.4 The worst-case behaviour of the simplex method, 139 11.5 The average running time of the simplex method, 142 11.6 The revised simplex method, 147

11.7 The dual simplex method, 148

12 Primal-dual, elimination, and relaxation methods 151

12.1 The primal-dual method, 151

12.2 The Fourier-Motzkin elimination method, 155 12.3 The relaxation method, 157

13 Khachiyan’s method for linear programming 163

13.1 Ellipsoids, 163

13.2 Khachiyan’s method: outline, 165 13.3 Two approximation lemmas, 166 13.4 Khachiyan’s method more precisely, 168

13.5 The practical complexity of Khachiyan’s method, 170 13.6 Further remarks, 171

14 The ellipsoid method for polyhedra more generally 172

14.1 Finding a solution with a separation algorithm, 172 14.2 Equivalence of separation and optimization, 177 14.3 Further implications, 183

15 Further polynomiality results in linear programming 190

15.1 Karmarkar’s polynomial-time algorithm for linear programming, 190 15.2 Strongly polynomial algorithms, 194

15.3 Megiddo’s linear-time LP-algorithm in fixed dimension, 199 15.4 Shallow cuts and rounding of polytopes, 205

Notes on polyhedra, linear inequalities, and linear

programming 209

(11)

Contents X

PART IV: INTEGER LINEAR PROGRAMMING

221

16 Introduction to integer linear programming 229

16.1 Introduction, 229

16.2 The integer hull of a polyhedron, 230 16.3 Integral polyhedra, 23 1

16.4 Hilbert bases, 232 16.5 A theorem of Doignon, 234

16.6 The knapsack problem and aggregation, 235 16.7 Mixed integer linear programming, 236

17 Estimates in integer linear programming 237

17.1 Sizes of solutions, 237

17.2 Distances of optimum solutions, 239

17.3 Finite test sets for integer linear programming, 242 17.4 The facets of P , , 243

18 The complexity of integer linear programming 245

18.1 ILP is .Jl.’9-complete, 245

18.2 .,V.V-completeness of related problems, 248

18.3 Complexity of facets, vertices, and adjacency on the integer hull, 251 18.4 Lenstra’s algorithm for integer linear programming, 256

18.5 Dynamic programming applied to the knapsack problem, 261 18.6 Dynamic programming applied to integer linear programming, 264

19 Totally unimodular matrices: fundamental properties and examples 266

19.1 Total unimodularity and optimization, 266 19.2 More characterizations of total unimodularity, 269 19.3 The basic examples: network matrices, 272 19.4 Decomposition of totally unirnodular matrices, 279

20 Recognizing total unimodularity 282

20.1 Recognizing network matrices, 282 20.2 Decomposition test, 287

20.3 Total unimodularity test, 290

21 Further theory related to total unimodularity 294

21.1 Regular matroids and signing of {O. 1)-matrices, 294 21.2 Chain groups, 297

21.3 An upper bound of Heller, 299

21.4 Unimodular matrices more generally, 301 2 1.5 Balanced matrices, 303

22 Integral polyhedra and total dual integrality 309

22.1 Integral polyhedra and total dual integrality, 310 22.2 Two combinatorial applications, 312

22.3 Hilbert bases and minimal TDI-systems, 315 22.4 Box-total dual integrality, 317

(12)

Contents xi 22.6 An integer analogue of Caratheodory's theorem, 326

22.7 Another characterization of total dual integrality, 327

22.8 Optimization over integral polyhedra and TDI-systems algorithmically, 330 22.9 Recognizing integral polyhedra and total dual integrality, 332

22.10 Integer rounding and decomposition, 336

23 Cutting planes 339

23. I Finding the integer hull with cutting planes, 339 23.2 Cutting plane proofs, 343

23.3 The number of cutting planes and the length of cutting plane proofs, 344 23.4 The Chvatal rank, 347

23.5 Two combinatorial illustrations, 348 23.6 Cutting planes and A"9-theory, 351 23.7 Chvatal functions and duality, 353 23.8 Gomory's cutting plane method, 354

24 Further methods in integer linear progamming 360

24.1 Branch-and-bound methods for integer linear progamming, 360 24.2 The group problem and corner polyhedra, 363

24.3 Lagrangean relaxation, 367

24.4 Application: the traveling salesman problem, 370 24.5 Benders' decomposition, 371

24.6 Some notes on integer linear programming in practice, 372

Historical and further notes on integer linear programming 375

Further notes on integer linear programming, 378

References 381

Notation index 452

Author index 454

(13)

(14)

Introduction and preliminaries

After the introduction in Section 1 . 1 , we discuss general preliminaries (Section 1.2), preliminaries on linear algebra, matrix theory and Euclidean geometry (Section 1.3), and on graph theory (Section 1.4).

1.1. INTRODUCTION

The structure of the theory discussed in this book, and of the book itself,

may be explained by the following diagram.

Part

r

Linear Algebra

linear equations J

Integer Linear Algebra Linear Programming

/inear insqua/ities

Np-complete

Integer Linear Programming

In Part I, ‘Linear Algebra’, we discuss the theory of linear spaces and of

systems of linear equations, and the complexity of solving these systems. The theory and methods here are to a large extent standard, and therefore we d o not give an extensive treatment. We focus on some less standard results, such as sizes of solutions and the running time of the Gaussian elimination method. It is shown that this method is a polynomial-time method, i.e. its running time is bounded by a polynomial in the size of the input data.

(15)

2 1 Introduction and preliminaries

In Part 11, ‘Lattices and Linear Diophantine Equations’, our main problem is

to solve systems of linear diophantine equations, i.e. to solve systems of linear

equations in integer variables. The corresponding geometric notion is that of a lattice. The existence of solutions here is characterized with the help of the Hermite normal form. One linear diophantine equation can be solved in poly-

nomial time with the classical Euclidean algorithm. More generally, also systems

of linear diophantine equations can be solved in polynomial time, with methods due to Frumkin and Votyakov, von zur Gathen and Sieveking, and Kannan and Bachem.

Also in Part I1 we discuss the problem of diophantine approximation. The

continued fraction method approximates a real number by a rational number

with low denominator, and is related to the Euclidean algorithm. Its extension to more dimensions, i.e. approximating a real vector by a rational vector whose

entries have one common low denominator, can be done with Lovasz’s basis

reduction method for lattices. These techniques are also useful in linear and

integer programming, as we shall see in Parts I11 and IV.

In Part 111, ‘Polyhedra, Linear Inequalities, and Linear Programming’, our main problems are the following:

(1) -solving systems of linear inequalities;

-solving systems of linear equations in nonnegative variables;

-solving linear programming problems.

These three problems are equivalent in the sense that any method for one of them directly yields methods for the other two. The geometric notion corres-

ponding to the problems is that of a polyhedron. Solutions of the problems (1)

are characterized by Farkas’ lemma and by the Duality theorem of linear

programming.

The simplex method is the famous method for solving problems (1); it is fast

in practice, and polynomial-time ‘on the average’, but no version of it could be proved to have polynomially bounded running time also in the worst case. It was for some time an open problem whether the problems (1) can be solved in polynomial time, until in 1979 Khachiyan showed that this is possible with the

ellipsoid method. Although it does not seem to be a practical method, we spend

some time on this method, as it has applications in combinatorial optimization. We also discuss briefly another polynomial-time method, due to Karmarkar.

The problems discussed in Parts 1-111 being solvable in polynomial time, in

Part IV ‘Integer Linear Programming’ we come to a field where the problems

in general are less tractable, and are MP-complete. It is a general belief that

these problems are not solvable in polynomial time. The problems in question are:

(2) -solving systems of linear diophantine inequalities, i.e. solving linear

(16)

3 1.2 General preliminaries

-solving systems of linear equations in nonnegative integer variables;

-solving integer linear programming problems.

Again, these three problems are equivalent in the sense that any method for one of them yields also methods for the other two. Geometrically, the problems correspond to the intersection of a lattice and a polyhedron. So the problems

discussed in Parts I1 and I11 meet here.

The theory we shall discuss includes that of characterizing the convex hull

P , of the integral vectors in a polyhedron P. The case P = P , generally gives rise to better-to-handle integer linear programming problems. This occurs when

P is defined by a totally unimodular matrix, or, more generally, by a totally duul integral system of inequalities. Inter alia, we shall discuss (but not prove) a deep theorem of Seymour characterizing total unimodularity.

If P is not-necessarily equal to P,, we can characterize P , with the cuttiny

plane method, founded by Gomory. This method is not a polynomial-time

method, but it yields some insight into integer linear programming. We also

discuss the result of Lenstra that for each fixed number of variables, the problems (2) are solvable in polynomial time.

The theory discussed in Part IV is especially interesting for combinatorial

optimization.

Before Parts I-IV, we discuss in the present chapter some preliminaries, while in Chapter 2 we briefly review the complexity theory of problems and algorithms. In particular, we consider polynomiality as a complexity criterion.

1.2. GENERAL PRELIMINARIES

Some general notation and terminology is as follows. If z is a real number, then

(3)

denote the lower integer part and the upper integer part, respectively, of a.

The symbols Z,Q, and [a denote the sets of integers, rationals, and real

numbers, respectively. Z + , Q + and 52, are the restrictions of these sets to the

nonnegatives. We denote, for real numbers IX and /j,

(4) a [ / ?if and only if o! divides

p,

i.e. if and only if

/?

= E,a for some integer

A.

Moreover, o! /?(mody ) means y ( ( a-

p).

If a,, ...,x, are rational numbers, not all equal to 0, then the largest rational

number y dividing each of z , , . . .,q Iexists, and is called the yrvurest common

divisor or g.c.d. of a , , . . .,x,, denoted by

(so the g.c.d. is always positive). The numbers x , , ... ,a, are relatiuely prime if

g.c.d. ( a I , ... ,a,,} = 1.

We write*f(x) = O(y(x)) for real-valued functionsf and g, if there exists a

(17)

4 1 Introduction and preliminaries

If we consider an optimization problem like

(6) max{cp(x)lxEAJ

where A is a set and <p:A+R, then any element x of A is called a feasible

solution for the maximization problem. If A is nonempty, problem (6) is called

,feasible,otherwise infeasible. Similarly, a set of conditions isfeasible (or solvable) if the conditions can be fulfilled all at the same time. Otherwise, they are called

infeasible (or unsolvable).Any instance satisfying the conditions is called afeasible solution.

If the maximum (6) is attained, we say that the maximum exists, isfinite, or

is bounded. A feasible solution attaining the maximum is called an optimum (or

optimal) solution. The maximum value then is the optimum value.

Similar terminology is used for minima.

A constraint is valid for a set S if each element in S satisfies this constraint.

‘Left-hand side’ and ‘right-hand side’ are sometimes abbreviated to L H S

and R H S .

1.3. PRELIMINARIES FROM LINEAR ALGEBRA, MATRIX THEORY, AND EUCLIDEAN GEOMETRY

We assume familiarity of the reader with the elements of linear algebra, such as linear (sub)space, linear (in)dependence, rank, determinant, matrix,

non-singular matrix, inverse, etc. As background references we mention Birkhoff

and Mac Lane [19771, Gantmacher [19591, Lancaster and Tismenetsky [19851,

Lang [1966a1, Nering [19631, Strang [1980).

If a = ( R ~ ,.. . ,a,) and b =

(PI,.

. . ,

b,)

are row vectors, we write a

<

b if a,

<

p i

for i = 1 , . . .,n. Similarly for column vectors. If A is a matrix, and x , b,y , and c

are vectors, then when using notation like

(7) A x = b , A x < b , ~ A = c

we implicitly assume compatibility of sizes of A , x , b , y , and c. So as for (7), if

A is an m x n-matrix, then x is a column vector of dimension n, b is a column

vector of dimension m, y is a row vector of dimension m, and c is a row vector

of dimension n.

Similarly, if c and x are vectors, and if we use

(8) cx

then c is a row vector and x is a column vector, with the same number of

components. So (8) can be considered as the inner product of c and x.

An n-vector is an n-dimensional vector.

If a is a row vector and /j is a real number, then a x =

P

and a x

<

are called

a linear equation and a linear inequalitj,, respectively. If vector x o satisfies a

linear inequality U.Y ,<

p,

then the inequality is called tight ( f o r x o )if a x , =

p.

If A is a matrix, and h is a column vector, we shall call A x = b a system qf

lineur ~quutions, and A x ,< b a syste,m of’ lineur inequulities. The matrix A is called the constraint matrix of the system.

(18)

5 1.3 Preliminaries from linear algebra, matrix theory, and Euclidean geometry

A system of linear inequalities can have several alternative forms, like (9) A x 3 h (for ( -A ) x

<

-h )

A x = h (for A x 6 h, -AY.

<

-h)

and so on.

If A'x

<

h' arises from A x d h by deleting some (or none) of the inequalities

in A x

<

b, then A ' x

<

b' is called a suhsystem of A x

<

h. Similarly for systems of linear equations.

The identity matrix is denoted by I , where the order usually is clear from

the context. If 6 is a real number, then an a l l 4 vector (all-6 inatri.~)is a vector

(matrix) with all entries equal to 6. So an all-zero and an ull-one vector have

all their entries equal to 0 and 1, respectively. 0 and 0 stand for all-zero vectors

or matrices, and 1 stands for an all-one vector, all of appropriate dimension.

The transpose of a matrix A is denoted by A T .We use

1 1 . 1 1

or

I/

9 I/ for the

Euclidvan norm, i.e.

d ( x , y )denotes the Euclidean distance of vectors ?c and y (i.e. d(x,j~):=

11

x -J'

11

2). and d ( x , P ) the Euclidean distance between x and a set P (i.e. d ( x , P ) : =

inf{d(x,Y)ly€P)).

The hull with centre x and radius p is the set

( 1 1) B ( x , p ) : = (Yld(.Y,Y)

<

P I .

A point XER" is an internal point of S C_ Iw" if there exists an E > 0 such that

(12) B ( X , & ) G S .

Other norms occurring in this text are the 1,- and the 1,-norms:

for x = . .

,t,)

_orx =

( t , , .

..,

4 ~ ~ .

An m x n-matrix A is said to have,fufl row rank (full column rank, respectively)

if rank A =m (rank A =n, respectively).

A row submatrix of a matrix A is a submatrix consisting of some rows of A.

Similarly, a column submatrix of A consists of some columns of A .

A matrix A =( a i j )is called upper triangular if z i j=0 whenever i >j . It is lower

triangular if x i j=0 whenever i < j . It is strictly upper triangular if crij=O whenever

ibj.It is strictly lower triangular if zij=O whenever i < j . It is a diagonal

matrix if aij =0 whenever i # j . The square diagonal matrix of order n, with the

numbers 6 , , . . . ,6, on its main diagonal, is denoted by

(19)

6 I Introduction and preliminaries

For any subset T of R, a vector (matrix) is called a T-uector (T-matrix) if its

entries all belong to 7'.A vector or matrix is called rational (integral, respectively)

if its entries all are rationals (integers, respectively).

A linear equation u x = or a linear inequality u x

<

[j is rational (integral) if

a and are rational (integral). A system of linear equations A x =h or inequalities A x

<

b is rational (integral) if A and b are rational (integral). A rational polyhedron

is a polyhedron determined by rational linear inequalities, i.e. it is (x~lw"1A x \< h }

for some rational system A x

<

h of linear inequalities.

Lattice point is sometimes used as a synonym for integral vector. A vector or matrix is l/k-integral if its entries all belong to ( l / k ) Z ,i.e. if all entries are integral multiples of l / k .

Scaling a vector means multiplying the vector by a nonzero real number.

For any finite set S, we identify the function x : S

-,

R with the corresponding

vector in Rs. If T E S , the incidence vector or characteristic vector of T is the

(0, 1)-vector in RS,denoted by

zT,

satisfying

If S and T are finite sets, an S x T-matrix is a matrix with rows and columns

indexed by S and T, respectively. If A is an S x T-matrix and h E [ W T , the product

Ah€WS is defined by:

for SGS(denoting A =(as,,)and h =

(p,)).

If % is a collection of subsets of a set S, the incidence matrix of %' is the

V x S-matrix M whose rows are the incidence vectors of the sets in W . So

M T , s= 1 if SET

(17)

M T q S=0 if s$T

for TEW,SES.

The support of a vector is the set of coordinates at which the vector is nonzero.

The linear hull and the u f i n e hull of a set X of vectors, denoted by 1in.hullX

and aff.hullX, are given by

( 1 8) lin.hull X = ( i l x l

+

* .

+

&Y,

1

t 3 0;xl,. . .,x,EX;R I , . . . ,;It E R >

aff.hullX= ( j L , x ,+ - + A , x , I t > l ; x l ,..., x , c X ; A 1,..., A,E(W;

2 ,

+-+A,

= I}.

A set C of vectors is conuex if it satisfies:

(19) if x , y ~ Cand 0

<

i,

<

1, then 2.u + ( I -E,)yEC.

The conuex hull of a set X of vectors is the smallest convex set containing

X , and is denoted by conv.hullX; so

(20) conv.hul1 X = ( l l x l

+ +

9.. h,x,lt 3 l ; x 1 , .. ., x , c X ;

(20)

7

1.3 Preliminaries from linear algebra, matrix theory, and Euclidean geometry

A (convex) cone is a nonempty set of vectors C satisfying

(21) if x, ~ E Cand A,p 30, then Ex

+

p y ~ C .

The cone generated hy a set X of vectors is the smallest convex cone containing X , and is denoted by cone X; so

. . ,ir

(22) cone X = (A1xl

+

.

-

+

A2xr]t2 0;xl,.

.

.,x,EX;il,.3O}.

If S E R", then a function f : S + R is convex if S is convex and f ( 1 x

+

(1 -A)y),< A f ( x )

+

(1 -A ) f ( y )whenever x , y ~ S and 0 d A ,< 1.

f

is concave if

-f

is convex.

Pivoting

If A is a matrix, say

where Q is a nonzero number, b is a row vector, c is a column vector, and D is a

matrix, then pivoting over the pivot element (1, 1) means replacing A by the matrix

Pivoting over any other element of A is defined similarly.

Some inequalities

We recall the following well-known (in)equalities (cf. Beckenbach and Bellman

[1983)). First the Cauchy-Schwarz inequality: if c, ER" then

(25) C T d

I1

c

It

*

tI

d

11.

If b , ,

.

,b, are column vectors in R", and B is the n x m-matrix with columns

b,, . . . ,b,, then

(26)

Jm

= the area of the parallelepiped spanned by b , ,

. .

.,b,. This implies the Hadamard inequality:

(27)

JZGG

d IIb, I I * * * * - J J b , , , ) ) .

In particular, if B is a square matrix, then

(28) ( d e t B (

<

Ilb, \ l * * . * * \ l b m J l *

(26)also implies that if A denotes the matrix with columns b , , .. .,b,-

,,

and

c is a vector orthogonal to b , , . . . ,b,- 1, where c is in the space spanned by

(21)

8 1 Introduction and preliminaries Positive definite matrices

A real-valued matrix D is called positive definite if D is symmetric and its eigenvalues all are positive. The following are equivalent for a real-valued symmetric matrix D:

(30) (i) D is positive definite;

(ii) D = CTCfor some nonsingular matrix C;

(iii) xTDx> 0 for each nonzero vector x.

Let D be a positive definite matrix, and let

D

= CTC,for nonsingular C. Two

vectors c and d are orthogonal (relative t o the inner product defined by D ) if

cTDd = 0. The norm

1.1

defined by D is given by

(31)

/ c / : = J X

By replacing b , , .

.

.,b, by C b , , .

..

,Cb,, c by (CT)-'c, and d by C d , the

(in)equalities (25), (27), and (29) become:

(32)

1

cTd

1 <

,/=-

,/m

(33)

JdetBTDB

<

Jh:Db,.

- -

Jm

det ATDA . Balls and ellipsoids

For S c R" and E 30, we denote

( 3 5 ) B ( S , E ) : =( y ~ [ w " IIlx-yll < E for some x in S}.

For XEIW", B(x,E):=B( {x},E) is the ball of radius c with centre x.

A set E of vectors in R" is called an ellipsoid if there exists a vector ZER" and

a positive definite matrix D of order n such that

(36) E=ell(z,D):= { ~ l ( x - z ) ~ D - ~ ( x - z ) < l}.

Here the parameters z and D are uniquely determined by E. The vector z is

called the centre of E. It follows from (30) that a set E is an ellipsoid if and

only if E is an affine transformation of the unit ball B(0,l).

1.4. SOME GRAPH THEORY

In this book we will describe occasionally some combinatorial applications, as illustrations of the theory. These applications are in terms of graphs.

Moreover, graphs will be used sometimes as a tool in some of the proofs and

algorithms, especially in Chapter 20 on totally unimodular matrices. Therefore, in this section we give a brief review of some elementary concepts, results, and problems in graph theory. The reader not familiar with graphs could read, for example, the first chapters of Wilson Cl9721 or Bondy and Murty [1976].

(22)

9 1.4 Some graph theory

Undirec ted graphs

An (undirected) graph is a pair G =(V,E), where V is a finite set, and E is a

family of unordered pairs of elements of V. The elements of V are called the

vertices or points of G, and the elements of E are called the edges or lines of G.

The term ‘family’ in the definition of graph means that a pair of vertices may

occur several times in E. A pair occurring more than once in E is called a

multiple edge. So distinct edges may be represented in E by the same pair.

Nevertheless, we shall often speak of ‘an edge ( U , w}’ or even of ‘the edge { U , w}’,

where ‘an edge of type ( U , w)’ would be more correct. Graphs without multiple

edges are called simple. Sometimes also loops are allowed, i.e. edges of the form

( 0 , U}.

We shall say that an edge ( U , w } connects the vertices U and w. The vertices

U and w are adjacent if there is an edge connecting U and w. The edge { U , w } is

said to be incident with the vertex U and with the vertex w, and conversely. The

vertices U and w are called the ends of the edge ( U , w } .

The number of edges incident with a vertex v is called the valency or degree of U , usually denoted by &(U). The maximum and minimum degree of the vertices of G are denoted by A(G) and 6(G).

The complementary graph of G , denoted by

c,

is the simple graph with the

same vertex set as G, and with edges all pairs {U,w ) of vertices which are not

in E . A simple graph is complete if E is the set of all pairs of vertices. The

complete graph with n vertices is denoted by K,. The line graph of G, denoted

by L(G),is the simple graph with vertex set E , in which two elements of E are

adjacent if and only if they intersect.

A graph G’ = (V’,E’) is a subgraph of G = ( V ,E ) if V’ c V and E’ C_ E. If E’ is

the family of all edges of G which have both ends in V’, then G’ is said to be

induced hy V’, and we denote G’ by ( V ’ ) .

Usually, the vertices of a graph are represented by dots, or small circles, in

the plane. Each edge, ( U , w } say, then is represented by a (curved) line segment

connecting the two dots representing U and w, and not meeting any other dot.

Two of such line segments may meet each other in or outside dots. If a graph can be represented in such a way that no two line segments meet each other outside dots, the graph is called planar. Two graphs which are nonplanar are

the graphs denoted by K , and K , , , , represented by:

K 3 , s

Figure 1

There is the famous characterization of Kuratowski [1930] that a graph is

planar if and only if it has no subgraph which comes from K , or from K,,,

(23)

10 I Introduction and preliminaries

A path in the graph G =(V,E)from U , to U , is a sequence of the form

(37) (U,,e,,u,,e,,...’U,-,,e,,u,)

where U , , . . . , U , are vertices and e , , . . .,er are edges, such that e, = { U , - 1, ui} for

i = 1,. ..,t. The vertices U , and U , are the starting point and the end point of the

path, respectively, or just the end points. We call path (37) a v,-U,-path, and it

is said to connect U , and U , . Path (37) is simple if all vertices and edges in (37)

are different.

The length of path (37) is t. The distance between two vertices r and s in a

graph is the minimum length of a path connecting r and s. Note that there is

an easy algorithm for finding the distance between r and s. It consists of recurs-

ively determining the sets V,, V , , V,, .. ., where V, is the set of vertices U of

distance i to r:

(38) (i) let Vo:= { r } ;

(ii) let V ,:={U E V\ { r }

I

U is adjacent to r } ;

(iii) if V,, V , , . . . ,

V i

have been determined, let

Vi+

,

:=

{ U E V \ ( V , u - . . u Vi)Ivis adjacent to at least one vertex in

Vi}.

Step (iii) is repeated until we have found a set V, containing s. Then d is the

distance between r and s, and an r-s-path of shortest length is easily derived

from V,, ...,V d . In the case where there is no path from r to s at all, this is

detected by arriving with (iii) at Vi+ =

0

without having s in V,u...u Vi.

Algorithm (38) is a ‘polynomial-time’ algorithm-see Section 2.4.

If U , =U , , path (37) is called closed. A closed path of length at least one and

without repeated edges or vertices (except for the end points) is called a circuit. An edge connecting two vertices of a circuit which are not connected by an

edge of the circuit is called a chord of the circuit.

A Hamilton circuit is a circuit containing each vertex of the graph exactly once (except for the end points). Famous is the following traueling salesman

problem: given a graph G =( V ,E ) and a ‘length’ function I:E Q+ , find a Hamilton circuit of minimum length. (The length of a circuit is the sum of the lengths of its edges.) The traveling salesman polytope is the convex hull of

the incidence vectors of the Hamilton circuits (so it is a subset of RE).

A graph is connected if each two vertices of the graph are connected by a

path. Connectedness by paths induces an equivalence relation on the vertices. Its classes are called the (connected) components of the graph. By extending

algorithms (38) it is easy to determine the components of a given graph G =(V,E )

‘in polynomial time’.

A forest is a graph having no circuits, and a tree is a connected forest. It is not difficult to see that the following are equivalent for a given simple graph G =( V ,E):

(39) (i) G is a tree;

(ii) G contains no circuits and / E (=

I

I/(-1;

(iii) G is connected and

I

El =

I

I/(-1 ;

(24)

1.4 Some graph theory 1 1

If we add one new edge connecting two vertices of the tree, we obtain a graph with a unique circuit. Each tree with at least two vertices has a vertex of degree one.

A subgraph G’ = (V’,E’) of G = ( V ,E ) is a spanning (&)tree of G if V’ = V

and G’ is a tree. Then G has a spanning subtree if and only if G is connected.

A maximal forest in G = ( V ,E) is a subgraph ( V ,E’) which is a forest, where E’

is as large as possible. This implies that ( V ,E’) has the same components as

( V ,E ) .

A clique (coclique, respectively) in the graph G = ( V ,E ) is a set of pairwise

adjacent (nonadjacent, respectively) vertices of G. The (maximum) clique problem

is the problem: given a graph G = ( V ,E ) , find a clique of maximum size. Similarly

for the (maximum) coclique problem. The clique polytope of G is the convex hull

of the incidence vectors of the cliques in G (so it is a subset of R‘). Similarly,

the coclique polytope of G is the convex hull of the incidence vectors of the

cocliques in G.

A matching is a set of pairwise disjoint edges. A matching covering all vertices is called a perfect matching or a l-factor. The matching polytope is the convex

hull of the incidence vectors of matchings (so it is a subset of

RE).

A graph G = (V,E ) is called bipartite if V can be partitioned into two classes

Vl and V , such that each edge of G contains a vertex in V , and a vertex in V,.

The sets Vl and V , are called colour classes. It is easy to see that a graph G is

bipartite if and only if G contains no circuits of odd length. Again by an extension

of algorithm (38)one can test in polynomial time whether a given graph G = (V,E )

is bipartite. If G is connected, choose an arbitrary vertex r, and determine the

classes V,, V , , V,, . . . as in (38), as long as they are nonempty. It is not difficult

to see that G is bipartite if and only if each of the classes V,, Vl, V 2 ,. . . is a

coclique. In that case, the sets V , U V 2U V , U .* . and V , U V , U V , U *

- -

form the

colour classes for G. If G is not connected, we can apply this algorithm to each

connected component of G.

A complete bipartite graph is a simple bipartite graph, with colour classes

say V , and V,, so that each vertex in V , is adjacent to each vertex in V,. A

complete bipartite graph with colour classes of sizes m and n is denoted by

K m , n(cf. Figure 1).

Let G =(V,E ) be a graph, and let e = (U,w } be an edge of G. Deletion of the

edge e means replacing G by G’:= ( V ,E\(e}). Contraction of the edge e means

replacing G by G’:= (V\(w}, E‘), where E’ consists of all edges of G contained

in V\ { w},together with all edges (x, U ) , where (x, w } is an edge of G different from e.

The adjacency matrix of an undirected graph G = ( V ,E ) is the matrix with

both rows and columns indexed by V, where the entry in position ( u , w ) is the

number of edges connecting U and w. So G is simple if and only if its adjacency

matrix is a (0,1)-matrix. The incidence matrix of G is the (0, 1)-matrix with

rows and columns indexed by V and E , respectively, where the entry in position

(25)

12 1 Introduction and preliminaries Directed graphs

A directed graph or a digraph is a pair D =( V ,A), where V is a finite set, and

A is a finite family of ordered pairs of elements of V. The elements of V are

called the vertices or points, and the elements of E are called the arcs of

D.

The

vertices U and w are called the tail and the head of the arc (v,w),respectively.

So the difference with undirected graphs is that orientations are given to the

pairs. Each directed graph gives rise to an underljJing undirected graph, in which

we forget the orientation of the arcs. Sometimes, when there is no fear of mis- understanding, we use ‘undirected’ terminology for directed graphs. The concepts of multiple, simple, and loop for directed graphs are analogous to those for undirected graphs.

We say that the arc ( U , w) enters w and leaves 21. If W is a set of vertices such

that u$W and WEW,then ( U , w) is said to enter W and to leave V\ W. If W G V,

then 6,(W) or X ( W )denotes the set of arcs in A entering W, and 6;(W) or

S’(W) denotes the set of arcs in A leaving W. 6 - ( u ) and 6 + ( u )stand for 6-( ( U } ) and 6+((0}).

A (directed) path, from u0 to U , , or a path, in a digraph D =( V ,A ) is a sequence of the form

(40) ( v o ,a,, U1 7 . * . 7 U , - 1, a,, 4)

where uo,. . . , U , are vertices and a , , . . . , a , are arcs, such that ai = ( u i -1, vi) for i = 1,.. . ,t. Path (40) is said to start in P~ and to end in U , . u0 is the starting

point and U , is the end point of the path. The number t is the length of the path

(40). A simple modification of algorithm (38) will give a polynomial-time

algorithm for finding a shortest r-s-path in a directed graph.

If ! : A+ R + , one easily extends this method to one finding a U,-v,-path

with length l ( a , )

+

. . .

+

/(a,)as small as possible (Dijkstra C19.591): Start with

wo:=u0 and do:= 0. If vertices wo,w , , ...,wk and numbers d o , d , , . ..,dk have

been found, choose arc ( w i ,w) attaining

(41) min(d,+ I ( w , , w ) ( i = O ,..., k ; w ~ V \ ( w ,,... , w k ) ; ( w i , w ) ~ A ) .

Let wk +

,

:=w, and let dk+ be the value of (41). By induction on k one shows

that dk is the length of a shortest uo-wk-path.

A u,-u,-path is closed if uo = U , . A directed circuit or a cycle is a closed path of

length at least one, without repeated vertices or arcs (except for its starting and

end point). A Hamilton cycle is a cycle containing all vertices of the digraph.

A set of arcs intersecting all r-s-paths for given vertices Y and s is called an

r-s-cut. So if r~ W, s# W then 6 , l ( W )is an r-s-cut.

Given a digraph D = ( V ,A) and vertices r,S E V ,a.floM?.fromr to s, or an r-s-flow,

(26)

13 1.4 Some graph theory

The second set of constraints here means that at any vertex U # r, s, the total

‘amount’ of flow entering U is equal to the total ‘amount’ of flow leaving u-the

j7ow conservation law. The value of the flow is the net amount of flow leaving r, i.e.

(43)

1

-

1

x ( 4 .

agh + ( r ) aeh - ( r )

It is not difficult to see that this is equal to the net amount of flow entering s.

Flow x is said to be subject to a given ‘capacity’ function c : A-+ R, if x(a)

<

c(a)

for each arc a.

A circulation is a flow for which the flow conservation law holds at each

vertex (i.e. it is an r-s-flow of value 0, or an r-r-flow, for arbitrary r and s).

An undirected path (undirected circuit, respectively) is a path (circuit, respec-

tively) in the underlying undirected graph. In a natural way, an undirected path

or circuit in a directed graph has forward arcs and backward arcs.

A digraph

D

= (V,A ) is called strongly connected if for each two vertices r

and s of

D

there is a directed r-s-path. D is (weakly) connected if its underlying

undirected graph is connected. A weakly connected digraph without undirected

circuits is called a directed tree.

The incidence matrix of a digraph

D

= (V,A ) is the matrix with rows and

columns indexed by I/ and A, respectively, where the entry in position ( v , a ) is

(27)

2 Problems, algorithms, and

complexity

The complexity of problems and algorithms in linear and integer programming will be one of the focuses of this book. In particular, we are interested in the solvability of problems in time bounded by a polynomial in the problem size.

Many of the problems in linear and integer programming, and in combinatorial optimization, can be easily seen to be solvable in finite time, e.g. by enumerating solutions. This generally yields an exponential-time algorithm. Solvability in polynomial time as a complexity criterion for problems was mentioned implicitly by von Neumann Cl9531 and explicitly by Cobham [1965] and Edmonds [1965a]. Edmonds introduced the term good for polynomial-time algorithms.

To indicate the significance of polynomial-time solvability, if we have a n algorithm with running time 2“ (where I I is the problem size), then a quadrupling of the computer

speed will add 2 to the size of the largest problem that can be solved in one hour, whereas if we have an algorithm with running time t i 2 , this size will be multiplied by 2.

Often, if a problem was proved to be solvable in polynomial time in theory, it could be solved quickly also in practice. Moreover, from a theoretical point of view it is an interesting phenomenon that most of the problems which can be solved in polynomial time also allow a certain representation in terms of polyhedra, and conversely. In this text some openings for an explanation of this phenomenon are described (e.g. the ellipsoid method, the primal-dual method, Theorem 18.3).

After the pioneering work by Cook Cl9711 and Karp [1972], one generally sets the polynomially solvable problems against the so-called 4’’.Y-complete problems. These problems can be proved to be the hardest among the problems in a certain natural class of problems, called MY-hardest, with respect to a certain natural complexity ordering. Although there is no proof, as yet, that .V.9-complete problems are really hard, no polynomial-time algorithms could be found to solve them, and it is a general belief that no such algorithms exist.

T o study problem complexity, we should describe more or less precisely what is meant by concepts like ‘problem’, ‘size’, ‘algorithm’, ‘running time’, etc. We shall however not venture upon defining each of these notions mathematically exactly, but will appeal at several points to the reader’s intuition, which will suftice to understand the greater part of this book. Here we confine ourselves to giving a brief introduction to complexity theory. For a more extensive and precise treatment, see Aho, Hopcroft, and Ullman [1974], Garey and Johnson [1979], and Savage [1976]. For an introduction, see Karp [19751.

(28)

15 2.2 Problems

2.1. LETTERS, WORDS, AND SIZES

Ground objects when formalizing problem complexity are symbols and strings

of symbols. Let C be a finite set (often C = (0, l}). C is called the alphabet and

its elements are called symbols or letters. An ordered finite sequence of symbols

from C is called a string (ofsymbols) or a word. C* stands for the collection of

all strings of symbols from C. The size of a string is the number of its components.

The string of size 0 is the empty string, denoted by

0,

Strings can have the form of (finite sequences of) rational numbers, vectors,

matrices, graphs, (systems of) linear equations or inequalities, and so on. There

are some standard ways of transformation in order to encode these objects

uniformly as proper strings of symbols from some fixed alphabet like (0,l).

Depending on the chosen transformation, this induces a concept of size for

these objects. To fix one (which will be used in this text), the sizes of a rational

number cc =p / q (where p and q are relatively prime integers), of a rational vector

c =(y,,.

.

.,y,) and of a rational matrix A =(aij)?=

,

,J= are:

(1) size(U):= 1

+

r i ~ g , ( i ~ i

+

1 ) i

+

rlog,(iqi

+

1 ) i

size(c):= n

+

size (7 ,)

+

-

*

+

size(y,) size( A ) : =rnn

+

1

size(aij).

i , j

The size of a linear inequality ax

<

p

or equation ax =

p

is equal to 1

+

size

( a )

+

size(p). The size of a system A x 6 b ( A x =h )of linear inequalities (equations)

is 1

+

size(A)

+

size(b). The size of a (directed or undirected) graph is equal to

the size of its incidence matrix.

(As said, our definition of size is one choice, convenient for our treatment-

other authors use other definitions. However, generally most of the size functions are ‘linearly equivalent’, in the sense that if size, and size, denote two size

functions, then size, =O(size,) and size, =O(size,).)

2.2. PROBLEMS

Informally, a problem can have the form of a question or a task. Mathe-

matically, a (search) problem is a subset ll of C* x C*, where C is some alphabet.

The corresponding metamathematical problem then is:

(2) given string Z E C * ,find a string y such that ( z , y ) ~ I l ,or decide that no

such string y exists.

Here the string z is called an instance or the input of the problem, and y is a

solution or the output.

Problem ll is called a decision problem or a yeslno problem if, for each ( z , y )

in I7, y is the empty string

0.

In that case, the problem is often identified with

the set 9of strings z in C* for which ( z , @ ) belongs to

n.

The problem, in

metamathematical language, can have forms like:

(3) ‘given string Z E C * ,decide whether z is in Y ’ ,or:

(29)

16 2 Problems, algorithms, and complexity

Examples of search problems are:

(4) (i) { ( ( A ,b),

@)I

A is a matrix, b is a column vector, such that A x

<

b for

at least one column vector x } ;

(ii) { ((A,b),x)(A is a matrix, b and x are column vectors, and A x

<

b ) ;

(iii) { ( ( A ,b,c),k )

I

A is a matrix, b is a column vector, c is a row vector, and

k = max{cx)Ax

<

b )>;

(iv) ( ( ( A ,b), 0 ) ) A x6 b is a system of linear inequalities with at least one

integral solution) ;

(v) { ( A , @ ) I Ais the adjacency matrix of an undirected graph with at least one perfect matching}.

Corresponding informal forms of these problems are:

( 5 ) (i) ‘Given a system A x d b of linear inequalities, does it have a

solution?’;

(ii) ‘Given a system A x \< b of linear inequalities, find a solution, if there

is one, or decide that no solution exists’;

(iii) ‘Given matrix A, column vector b, and row vector c, determine

max {cx

I

A x \< b}’;

(iv) ‘Given a system A x \< b of linear inequalities, does it have an integral

solution?’;

(v) ‘Given an undirected graph, does it have a perfect matching?’

2.3. ALGORITHMS AND RUNNING TIME

We assume some intuitive feeling on the part of the reader of what algorithms are, and how to operate with them (cf. Knuth C1968: Ch. 13). An algorithm is a list of instructions to solve a problem. Following Turing C1936-71, an algorithm can be formalized in terms of a Turing machine-see Aho, Hopcroft, and Ullman

Cl9741 or Garey and Johnson [1979]. In Section 18.1 we shall discuss the

concept of algorithm more formally.

For a given input ZEC*,an algorithm for problem

n

c

C*x

C*

determines an

output y such that ( z , y )is in

n,

or stops without delivering an output if there

exists no such y. An algorithm can have the shape of a computer program, which is formally just a finite string of symbols from a finite alphabet. Mathe-

matically, an algorithm can be defined as a finite string A of O S and 1’s. One

says that A solves problem Il or A is an algorithm for

n,

if for any instance z

of

n,

when giving the string ( A , z )to a ‘universal Turing machine’, the machine

stops after a finite number of steps, while delivering a string y with ( z , y ) ~ n ,or

delivering no string in the case where such a string y does not exist.

There are several ways of defining a concept of running time, to indicate the

number of ‘elementary bit operations’ in the execution of an algorithm by a