Linear Optimization. Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.

Linear Optimization

Yinyu Ye

Department of Management Science and Engineering Stanford University

Stanford, CA 94305, U.S.A.

Introduction to Linear Optimization

The field of optimization is concerned with the study of maximization and minimization of mathematical functions. Very often the arguments of (i.e., variables or unknowns in) these functions are subject to side conditions or constraints. By virtue of its great utility in such diverse areas as applied science, engineering, economics, finance, medicine, and statistics, optimization holds an important place in the practical world and the scientific world. Indeed, as far back as the Eighteenth Century, the famous Swiss mathematician and physicist Leonhard Euler (1707-1783) proclaimeda that . . . nothing at all takes place in the Universe in which some rule of maximum or minimum does not appear.

a_{See Leonhardo Eulero, Methodus Inviendi Lineas Curvas Maximi Minimive Proprietate Gaudentes,}

Linear Programs and Extensions (in Standard Form)

Linear Programming

(LP )

minimize

c

T

x

subject to

Ax = b,

x

≥ 0.

Linearly Constrained Optimization Problem

(LCOP )

minimize

f (x)

subject to

Ax = b,

Linear Complementarity Problem Find nonnegative vectors

x

≥ 0, s ≥ 0

such that

(LCP )

s = M x + q,

s

T

x = 0.

Rectified Linear Unit Operation:

s = max

{0, ay + b} ⇒ sx = 0, s = x + ay + b, (s, x) ≥ 0.

Conic Linear Programming

(CLP )

minimize

c

• x

subject to

a

_i

• x = b

_i

, i = 1, ..., m,

x

∈ K,

where

K

is a closed convex cone.

CLP: LP, SOCP, and SDP Examples

minimize

2x

₁

+ x

₂

+ x

₃ subject to

x

₁

+ x

₂

+ x

₃

= 1,

(x

1

; x

2

; x

3

)

≥ 0;

minimize

2x

₁

+ x

₂

+ x

₃ subject to

x

₁

+ x

₂

+ x

₃

= 1,

√

x

2₂

+ x

2₃

≤ x

1

.

minimize

2x

₁

+ x

₂

+ x

₃ subject to

x

_

1

+ x

2

+ x

3

= 1,



x

1

x

2

x

2

x

3



 ≽ 0,

LP Terminology

•

decision variable/activity, data/parameter

•

objective/goal/target, coefficient vector

•

constraint/limitation/requirement, satisfied/violated

•

equality/inequality constraint, direction of inequality, non-negativity

•

constraint matrix/the right-hand side

•

feasible/infeasible solution, interior feasible solution

•

optimizers and optimum values

Linear Programming Facts

•

The feasible region is a convex polyhedron.

•

Every linear program is either feasible/bounded, feasible/unbounded, or infeasible.

•

If feasible/bounded, every local optimizer is global and all optimizers form a convex polyhedron set.

•

All optimizers are on the boundary of the feasible region.

•

If the feasible region has an extreme point, then there must be an extreme optimizer.

Linear Optimization Model and Formulation

•

Sort out data and parameters from the verbal description

•

Define the set of decision variables

•

Formulate the linear objective function of data and decision variables

Transportation Problem

min

∑

m_i=1

∑

n_j=1

c

ij

x

ij s.t.

∑

n_j=1

x

_ij

= s

_i

,

∀i = 1, ..., m

∑

m i=1

x

ij

= d

j

,

∀j = 1, ..., n

x

ij

≥ 0, ∀i, j.

2

3

n

1

2

m

.

.

.

.

.

.

s1

s2

sm

d2

d3

dn

Max-Flow Problem

Given a directed graph with nodes

1, ..., m

and edges

A

, where node

1

is called source and node

m

is called the sink, and each edge

(i, j)

has a flow rate capacity

k

_ij. The Max-Flow problem is to find the largest possible flow rate from source to sink.

Let

x

_ij be the flow rate from node

i

to node

j

. Then the problem can be formulated as

maximize

x

_m1

subject to

∑

_j:(j,1)∈A

x

_j1

−

∑

_j:(1,j)∈A

x

_1j

+ x

_m1

= 0,

∑

j:(j,i)∈A

x

ji

−

∑

j:(i,j)∈A

x

ij

= 0,

∀i = 2, ..., m − 1,

∑

j:(j,m)∈A

x

jm

−

∑

j:(m,j)∈A

x

mj

− x

m1

= 0,

2

1

Sink

Source

6

3

3

4

5

3

4

4

2

3

7

3

Prediction Market I: World Cup Information Market

Order:

#1

#2

#3

#4

#5

Argentina

1

0

1

1

0

Brazil

1

0

0

1

1

Italy

1

0

1

1

0

Germany

0

1

0

1

1

France

0

0

1

0

0

Bidding Prize:

π

0.75

0.35

0.4

0.95

0.75

Quantity limit:

q

10

5

10

10

5

Order fill:

x

x

₁

x

₂

x

₃

x

₄

x

₅

Prediction Market II: Call Auction Mechanism

Given

m

potential states that are mutually exclusive and exactly one of them will be realized at the maturity. An order is a bet on one or a combination of states, with a price limit (the maximum price the participant is willing to pay for one unit of the order) and a quantity limit (the maximum number of units or shares the participant is willing to accept).

A contract on an order is a paper agreement so that on maturity it is worth a notional $

1

dollar if the order includes the winning state and worth $

0

otherwise.

Prediction Market III: Input Order Data

The

i

th order is given as

(a

_i·

∈ R

m₊

, π

_i

∈ R

₊

, q

_i

∈ R

₊

)

:

a

_i· is the betting indication row vector where each component is either

1

or

0

a

i·

= (a

i1

, a

i2

, ..., a

im

)

where

1

is winning state and

0

is non-winning state;

π

_i is the price limit for one unit of such a contract, and

q

_i is the maximum number of contract units the better like to buy.

Prediction Market IV: Output Order-Fill Decisions

Let

x

_i be the number of units or shares awarded to the

i

th order. Then, the

i

th bidder will pay the amount

π

i

· x

i and the total amount collected would be

π

T

x =

∑

i

π

i

· x

i.

If the

j

th state is the winning state, then the auction organizer need to pay the winning bidders

(

_n

∑

i=1

a

ij

x

i

)

= a

T_·j

x

where column vector

a

_·j

= (a

1j

; a

2j

; ...; a

nj

)

Prediction Market V: Worst-Case Profit Maximization

max

π

T

x

− max

j

{a

T_·j

x

}

s.t.

x

≤ q,

x

≥ 0.

max

π

T

x

− max(A

T

x)

s.t.

x

≤ q,

x

≥ 0.

This is NOT a linear program.

However, the problem can be rewritten as

max

π

T

x

− y

s.t.

A

T

x

− e · y ≤ 0,

x

≤ q,

x

≥ 0,

where

e

is the vector of all ones. This is a linear program.

max

π

T

x

− y

s.t.

A

T

x

− e · y + s

₀

= 0,

x + s

= q,