Mixed integer programming methods for supply chain optimization

Mixed

­

integer

 

programming

 

methods

 

for

 

supply

 

chain

 

optimization

C

 

h

 

r

 

i

 

s

 

t

 

o

 

s

  

T.

  

M

 

a

 

r

 

a

 

v

 

e

 

l

 

i

 

a

 

s

Chemical and Biological Engineering

University of Wisconsin, Madison, WI 53706, USA

Outline

1. Supply

 

Chain

 

Management

2. Linear

 

Programming

3. Basic

 

Mixed

­

integer

 

Linear

 

Programming

4. Advanced

 

Mixed

­

integer

 

Linear

 

Programming

5. Software

 

Overview

6. Combinatorial

 

Optimization

7. Chemical

 

Production

 

Scheduling

The

 

Chemical

Supply

 

Chain

Supply chain (SC):

network of facilities and distribution options for the procurement of materials; 

transformation of materials into intermediate and finished products; and 

distribution of these finished products to customers. 

The goal in supply chain management is to 

coordinate

 

materials,

 

information

 

and

 

financial

 

flows

to 

fulfill customer demands while improving competitiveness of the supply chain as a whole; in 

coordinating these flows various decisions have to be made.

Suppliers Manufacturing Facilities Warehouses/ dist. centers Retailers Customers

Chemical industry changes:

ƒ New products ⇒multiproduct (and batch?) processes ƒ Holistic view of operations

Tayur ‐ FOCAPO 2003:

ƒ Inventories in SCs: ~ $10 trillion (10% of US GDP) ƒ Same customer satisfaction levels with 50% inventory Traditional PSE is concerned with the  development of methods for the design,   operation, and control of chemical systems. 

Supply

 

Chain

 

Planning

 

Matrix

 

(Meyr et

 

al.,

 

2002)

procurement production distribution sales

Purchasing & Material Requirements Planning Distribution Planning Transport Planning Demand Planning Demand  Fulfillment Long‐term Medium‐term Short‐term Strategic Planning Master Planning Scheduling Production Planning

ƒ

Integration across time scales and 

functions

ƒ

Many 

new

interesting problems

ƒ

Power – industrial gases – steel SC 

ƒ

Oil – industrial gases – chemicals SC

under 

power

constraints

Refinery Industrial

Gases Chemicals Customers

Supply

 

Chain

 

Planning

 

Matrix

 

(Meyr et

 

al.,

 

2002)

Sugarcane Sugarcane Wood Wood Switch  grass Switch  grass Wood  waste Wood  waste Sugars Sugars Corn

Corn CornCorn  GrainGrain

Corn  Stover Corn  Stover Hydrolysis Hydrolysis Fractionation ‐Catalyst ‐Steam ‐Acid ‐Enzymes  Fractionation ‐Catalyst ‐Steam ‐Acid ‐Enzymes  Bio‐oils (Sugars,  Acids) & Lignin Bio‐oils (Sugars,  Acids) & Lignin Lignin Lignin Demethoxylation BTXBTX Hydrodeoxygenation Phenols  &BTX Phenols  &BTX Fast  pyrolysis Fast  pyrolysis Liquefaction Liquefaction Hydrolysis Hydrolysis Bagasse Bagasse Fermentation Fermentation Dehydration Dehydration Succinic acid Succinic acid Fumaric acid Fumaric acid Malic acid Malic acid Furfural Furfural Dehydration

Dehydration LevulinicLevulinic_acidacid FDCA FDCA

C4

C5

C6 procurement production distribution sales

Purchasing & Material Requirements Planning Distribution Planning Transport Planning Demand Planning Demand  Fulfillment Long‐term Medium‐term Short‐term Strategic Planning Master Planning Scheduling Production Planning

ƒ

Integration across time scales and 

functions

ƒ

Many 

new

interesting problems

ƒ

Power – industrial gases – steel SC 

ƒ

Oil – industrial gases – chemicals SC

under 

power

constraints

Supply

 

Chain

 

Planning

 

Matrix

 

(Meyr et

 

al.,

 

2002)

ƒ

Integration of production planning and scheduling 

ƒ Exxon Mobil: 2% reduction in operating costs, 20% inventory reduction (Shobrys & White, 2002)

ƒ DuPont:  Capital tied up in inventory reduced from $165 to $90 million (Shobrys & White, 2002)

procurement production distribution sales

Purchasing & Material Requirements Planning Distribution Planning Transport Planning Demand Planning Demand  Fulfillment Long‐term Medium‐term Short‐term Strategic Planning Master Planning Scheduling Production Planning

ƒ

Integration across time scales and 

functions

ƒ

Many 

new

interesting problems

ƒ

Power – industrial gases – steel SC 

ƒ

Oil – industrial gases – chemicals SC

under 

power

constraints

Outline

1. Supply

 

Chain

 

Management

2. Linear

 

Programming

3. Basic

 

Mixed

­

integer

 

Linear

 

Programming

4. Advanced

 

Mixed

­

integer

 

Linear

 

Programming

5. Software

 

Overview

6. Combinatorial

 

Optimization

7. Chemical

 

Production

 

Scheduling

Linear

 

Programming

Standard optimization problem:

max f(x) n - number of variables

s.t. g(x) = 0 m - number of equality constraints h(x)≤ 0 l – number of inequalities

x∈X

If functions f, g, h are linear and variables x are continuous, we have a Linear Programming (LP) Model.

Notes:

1. Free z∈ℜ variables can be replaced by two nonnegative variables x, y≥ 0:

z = x – y

2. Inequalities can be transformed to equalities with slack variables x1 + 2x2 ≤ 10 ⇒ x1 + 2x2 + s1 = 10, s1 ≥ 0

3. Minimization problems can be expressed as maximization problems: min f(x) ⇔ max –f(x)

Thus, we can use the standard LP form: (LP) max z = cTx

s.t. Ax = b x≥ 0

where x: (n+l)-vector; A:(m+l)×(n+l) matrix; c: (n+l)-vector; b: (m+l)-vector

LP

 

Geometry

A process is used for the production of products P1 and P2 from raw materials A and B according to: 2A + B

→

3P1 (1)

A + 2B

→

3P2 (2)

If the maximum availability of raw materials A and B is 10 kg/hr, the price of both chemicals is $100/kg, and the demand for products P1 and P2 is 12 and 13 kg/hr, respectively, what is the optimal production mix?

The problem can be represented as the following network (the labels refer to the streams, not species): A B P2 P1 S_A S_B S_A1 S_A2 S_B2 S_B1 Reaction 1 Reaction 2 S₁ S₂

If FP1 and FP2 are the amounts of products P1 and P2,

respectively, the optimization problem (P1) can be stated as follows: 13 0 , 12 0 30 2 30 2 . 100 100 max 2 1 2 1 2 1 2 1 ≤ ≤ ≤ ≤ ≤ + ≤ + + = P P P P P P P P F F F F F F st F F Z (P1) F_P2 2F_P1+ F_P2≤30 F_P1+ 2F_P2≤30 F_P1≤12 F_P2≤13 30 20 10

LP

 

Geometry

x₁ x₂ 2x₁+ x₂+ s₁= 30 x₁+ 2x₂+ s₂=30 x₁+ s₃=12 x₂+ s₄=13 10 20 30 30 20 10 max 10x₁+10x₂ s.t 2x₁+ x₂+ s₁ = 30 x₁+2x₂ + s₂ = 30 x₁ + s₃ = 12 x₂ + s₄ = 13 x₁, x₂, s₁, s₂, s₃, s₄≥0 max 10x₁+10x₂ s.t 2x₁ + x₂ ≤30 x₁+2x₂ ≤30 x₁ ≤12 x₂ ≤13 x₁, x₂≥0

Definition: A basic solution to an (LP) is determined by fixing (n-m)

variables to 0, and solving the (m×m) system to obtain the remaining.

•Variables fixed at zero are non-basicvariables.

•Variables obtained by solving the equalities are basic.

Definition: A basic feasible solution to an (LP) is a basic solution that satisfies nonnegativity constraints.

Definition: x∈Fis an extreme point iff x= αx1_{+ (1-}α_{) x}2 _and α∈_{(0,1), x}1_{, x}2_∈F

implies x1_{= x}2_.

i.e. xcan not be obtained by linear combination of x1_{, x}2_{∈F: x}1_≠x2_.

⇒The basic feasible points of an LP are the extreme point of its feasible region.

Let A = [B | N], _⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = N B x x

x where B: m×m full-rank, N: m×(n-m), xB: m-vector and xN: (n-m)-vector

Ax = BxB + NxN = b

If xN= 0 ⇒ xB = B-1b - B-1NxN Basic point

z = cBxB + (-cBT B-1N + cNT)xN Objective function

Theorem 1: A point xin Fis an extreme point iffx_B= B­1_b≥0, x N= 0 Theorem 2: The optimal solution x*_{of (LP) lies at an extreme point of F.} Theorem 3: Point x*_{is optimum if Reduced Cost RC = ‐c}

BTB­1N+ cNT≤0

Basic solution

Simplex

 

Method

Main idea of Simplex: If current basis non‐optimal, swap a non‐basic variable for a basic one until (‐c_BT_B‐1_N + c NT) ≤0T x₁ x₂ (0,13) (4,13) (10,10) (6,12) (0,12) max 10x₁+10x₂ s.t 2x₁+ x₂+ s₁ = 30 x₁+2x₂ + s₂ = 30 x₁ + s₃ = 12 x₂ + s₄ = 13 x₁, x₂, s₁, s₂, s₃, s₄≥0 we start at x1=x2=0 ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ 1 0 0 0 1 0 0 1 0 0 0 1 0 0 1 0 2 1 0 0 0 1 1 2 M M M M N B x_B= B‐1_b z= c_Bx_B+ (‐c_BT_B‐1_N+ c NT)xN x_N= [x₁, x₂] = [0, 0] x_B= [s₁, s₂, s₃, s₄] = [30, 30, 12, 13] z= 0 + 10x₁+ 10x₂ x_N= [s₃, x₂] = [0, 0] x_B= [s₁, s₂, x₁, s₄] = [6, 18, 12, 13] z= 120 + 10x₂– 10s₃ x_N= [s₃, s₁] = [0, 0] x_B= [x₂, s₂, x₁, s₄] = [6, 6, 12, 7] z= 180 + 10s₃– 10s₁ x_N= [s₂, s₁] = [0, 0] x_B= [x₂, s₃, x₁, s₄] = [10, 2, 10, 3] z= 200 – (10/3)s₂– (10/3)s₁

ƒ

Simplex algorithm developed by G. Dantzig (1947)

ƒ

Very popular due to wide range of applications formulated using LP

ƒ

Considered as one of the most important algorithms in the 20

th

_century

Simplex

 

Method

 ­

Remarks

Initialization

It is not always easy to find an initial basis.

For instance, if after adding the slack variable we have some components of b that are negative.

In this case we apply Simplex in two phases.

Phase I: To each equality

∑

= j

i j ijx b

a _{with b}

i<0, subtract an artificial variable ui, add constraint ui≥ 0 and solve:

min Σui

s.t. Ax = b -u u≥ 0

The original LP has a feasible solution iff the auxiliary LP has an optimal solution with objective value equal to zero.

Phase II: Use the last basis of phase I as a starting basis for Phase II, after you delete artificial variables ui and change

objective function. Infeasible Æ

Termination

It is not guaranteed that a feasible LP has an optimal solution.

The problem can be unbounded. The simplex method detects such programs and returns a “proof” of unboundedness. ⇒ When checking to determine the leaving variable, no constraint becomes binding.

Degeneracy

ƒ A basic solution is degenerate if a basic variable is equal to zero.

ƒ In this case, the Simplex method may go through many iterations without increasing the value of the objective function, and the same basis is encountered again (Cycling).

Duality

 ­

I

Given a primal LP problem:

min cT x (LP-P) s.t. Ax = b

x

≥

0

the corresponding dual LP problem is formulated as: min bT y (LP-D)

s.t. ATy

≤

c

How do we construct (LP-D) from (LP-P)?

ƒ For each constraint of (LP-P), there is a variable in (LP-D)

ƒ For each variable of (LP-P), there is a constraint in (LP-D)

ƒ If (LP-P) is min problem, the dual is max.

ƒ If the constraints of (LP-P) are equalities (inequalities) the variables of (LP-D) are unconstrained (constrained)

ƒ If the variables of (LP-P) are free (non-negative), the constraints of (LP-D) are equalities (inequalities)

ƒ Non-negative variables result in ≤ inequalities in max problems

ƒ Non-negative variables result in ≥ inequalities in min problems

Duality

 ­

II

Let FP and FD be the feasible region of (LP-P) and (LP-D), respectively, we introduce the following convention:

ƒ If FP _{= ∅ =+∞} ∈ c x T F xinfP ƒ If FD _{= ∅ =−∞} ∈ y bT F y D sup

Theorem 1: For any (LP): c x bTy

F y T F

xinf∈ P ≥ sup_∈ D [Proof]

Theorem 2: If either FP≠ 0 or FD≠ 0, then c x bTy F y T F x∈ P _∈ D = sup inf _[Proof]

⇒ if one of these problems is solvable, then so is the other and c x bTy F y T F xmin_∈ P =max_∈ D Primal Dual

Optimal Infeasible Unbounded

Optimal √ - -

Infeasible - √ √

Duality

 ­

III

Sign of dual variables

ƒ Why equality constraints, have unrestricted dual variables?

ƒ What does the sign say about the RHS?

Nonnegative variables in (LP-P) result in ≤ inequalities in max problems

≥ inequalities in min problems

Furthermore, the dual solution provides us with

sensitivity

information

ƒ

The dual variable corresponding to a primal constraint, indicates whether this constraint is active;

and

by how much

ƒ

Dual values = shadow prices = marginal values

Thus, we can solve (LP-D) instead of (LP-P) if the first is easier.

ƒ

Computationally, the number of constraints is more important than the no of variables

Duality

 ­

IV

Re-optimization

Let assume that we have solved to optimality (LP-P) to obtainz* = cTx* = bTy*

ƒ What happens if we add a new constraintam+1 x = bm+1 to (LP-P)?

⇒ (LP-P) may become infeasible.

ƒ What happens to (LP-D)?

We add a variable; if we set ym+1= 0, we still have a feasible solution ⇒ We can use the existing basis to continue.

Dual Simplex Method

In general, a feasible non-optimal point of (LP-P) corresponds to an infeasible point of (LP-D) [Why???]. ƒ In the simplex algorithm, we try to reduce infeasibility of the dual problem.

ƒ Dual simplex method

Commercial solvers use there three powerful LP algorithms:

ƒ Primal Simplex

ƒ Dual Simplex

ƒ Barrier method

ƒ

Different methods perform better in different problems

ƒ

Commercial solvers detect certain problem structures & automatically choose appropriate method

Outline

1. Supply

 

Chain

 

Management

2. Linear

 

Programming

3. Basic

 

Mixed

­

integer

 

Linear

 

Programming

4. Advanced

 

Mixed

­

integer

 

Linear

 

Programming

5. Software

 

Overview

6. Combinatorial

 

Optimization

7. Chemical

 

Production

 

Scheduling

MIP

 

in

 

Chemical

 

Engineering

Many decisions are discrete in nature:

Process Design:

9

Selection of equipment units (e.g. distillation vs. extraction) ‐

Superstructure

 

Optimization

9

Selection of equipment sizes

9

Matching in heat exchanger networks (hot 

↔

cold pairs)

Process operations:

9

Decision to expand/retrofit a process

9

Split of customer orders to batches

9

Assignment of batches to equipment units

9

Sequencing of batches in the same unit

Process Control:

9

Selection of linearized model for MPC

9

On/off control action

Applications: Systems Biology

9

Gene addition and/or knock‐out 

Discrete

 

Decisions

 

in

 

Process

 

Design

1. Separate HCl/C₂H₃Cl – C₂H₄Cl₂ 2. Separate HCl – C₂H₃Cl 1. Separate HCl – C₂H₃Cl/C₂H₄Cl₂ 2. Separate C₂H₃Cl – C₂H₄Cl₂ Oxygen Air Ethylene Chlorine Vinyl Chloride Hydrogen Chloride Ethylene Dichloride Ethylene Dichloride Water Flash Direct Chlorination Oxychlorination Low P High P Purge Hydrogen Chloride Reactions: Direct chlorination C₂H₄+ Cl₂→C₂H₄Cl₂ Oxychlorination C₂H₄+ 2HCl + 1/2O₂→C₂H₄Cl₂+ H₂O Pyrolisis C₂H₄Cl₂→C₂H₃Cl + HCl Separation:HCl/C₂H₃Cl/C₂H₄Cl₂mixture

Example:

What is the best way to produce Vinyl Chloride Monomer?

Optimization Decisions:

ƒ

Distillation sequence (discrete), sizing (discrete or continuous), operating conditions (continuous)

Discrete variables are also used to:

ƒ

Represent logic conditions

ƒ

Enforce set of constraints (often including only continuous variables)

Discrete

 

Decisions

 

in

 

Process

 

Operations

Products 1st_{stage 2}nd_{stage 3}rd_stage

(Batches) Reaction Separation1 Separation2

Example:

Scheduling of a multi-product, multi-stage plant

Optimization Decisions:

ƒ

Number of batches

ƒ

Assignment of batches to units

ƒ

Sequencing of batches in a unit

Basic

 

Integer

 

Programming

 

Problems

Knapsack Problem

ƒ We are given n items to be packed in a knapsack

ƒ Each item i∈{1, 2, …n} has a value p_iand a weight w_i

ƒ The capacity of the knapsack is W

ƒ The goal is to choose the items that fit in the knapsack with the max value We define:

X_i= 1 if item iis chosen

If we had multiple resources j∈{1, 2, ..m} (e.g. weight, volume, etc.) with capacities W_j we would replace (K.2) with (K.3):

j

W

X

w

_j i i ij

≤

∀

∑

∑

=

j i i i

X

p

z

,

max

Objective:

W

X

w

i i i

≤

∑

Capacity constraint

i

X

_i

∈

{

0

,

1

}

∀

(K.1)

(K.2)

(K.3)

Items Knapsack . . . 1 2 n

Basic

 

Integer

 

Programming

 

Problems

Assignment Problem

ƒ We are given n jobs and n machines (classes/instructors, routes/buses)

ƒ Each job i∈{1, 2, …n} has to be assigned to a machine j∈{1, 2, … n}

ƒ The cost of assignment is c_ij

ƒ The objective is to find the assignment with the minimum total cost We define:

X_ij= 1 if job iis assigned to machine j

∑

=

j i ij ij

X

c

z

,

min

Objective:

_(A.1)

i

X

j ij

=

∀

∑

1

(A.2)

Every job is assigned

j

X

i

ij

=

∀

∑

1

Exactly one job per machine

j

i

X

_ij

∈

{

0

,

1

}

∀

,

(A.3)

ƒ If solved as LP, there is always an optimal solution with integral values ƒ Studied by  Carl Gustav Jacobi (1890) ƒ Solved effectively by the Hungarian method (Kuhn, 1955) ƒ Improved by Munkres (1957) Jobs Machines . . . . . . 1 2 n 1 2 n

Basic

 

Integer

 

Programming

 

Problems

Cities

Traveling Salesman Problem

There are

n

cities to be visited exactly once by a traveling salesman

The distance/cost from city

i

to city

j

is

c

_ij

The objective is to find the sequence of cities that yield the minimum

total distance/cost

We define:

X

_ij

= 1 if we visit city

j

after city

i

Exercise

ƒ

Are constraints (TSP.2) & (TSP.3) enough?

ƒ

What type of solution can we get?

ƒ

How can we improve our model?

∑

=

j i ij ij

X

c

z

,

min

Objective:

_(TSP.1)

j

X

i ij

=

∀

∑

1

_(TSP.2)

_{There is an incoming arc for every city j}

j

i

X

_ij

∈

{

0

,

1

}

∀

,

i

X

j ij

=

∀

Basic

 

Integer

 

Programming

 

Problems

Set Covering Problem

ƒ We are given n tasks to be covered by m<nmachines (flights/crew)

ƒ Each machine j∈{1, …m} can carry out a subset of tasks I_j⊂I = {1, 2, …n}

ƒ The compatibility is represented via parameter: α_ij= 1if i∈I_j ƒ The cost of a machine is c_j

ƒ The objective is to find the set of machines that coversall tasks and has the minimum cost

We define: X_j= 1 if machine jis chosen

∑

=

j j j

X

c

z

min

Objective:

(SC.1)

i

X

a

j j ij

≥

∀

∑

1

Every task is covered

j

X

_j

∈

{

0

,

1

}

∀

(SC.1)

Jobs Machines . . . . . . 1 2 n 1 2 m

Basic

 

Mixed

­

integer

 

Programming

 

Problems

Facility Location Problem – I

There are n (existing + new) facilities (plants) and

m

customers

We want to decide how to satisfy customer demand

What new facilities to build, how much to produce and ship

The capacity (cost of setting up) of facility

i

∈

{1, 2, …n}

is

α

_i

(

f

_i

)

The cost of shipping material from

i

to j is

c

_ij

The demand of customer

j

∈

{1, 2, …m}

is

d

_j

We define:

X

_i

= 1

if we set up (install) facility

i

Y

_ij

= amount shipped from

i

to

j

Facilities Customers . . . . . . 1 2 n 1 2 m

c

_ij

d

_j

a

_i

, f

_i

∑

∑

+

=

j i ij ij i i i

X

c

Y

f

z

,

min

Objective:

_(FLI.1)

Minimize set-up and shipping costs

More realistic problem

:

9

There is also production cost

p

_i

, transportation set-up cost

t

_ij

, and transportation capacity

s

_ij

9

The problem has to be solved over multiple planning periods

j

d

Y

_j

i

ij

≥

∀

∑

_(FLI.2)

Customer demand satisfaction

j

i

Y

i

X

_i

∈

{

0

,

1

}

∀

,

_ij

≥

0

∀

,

i

X

a

Y

_{i i} j ij

≤

∀

Basic

 

Mixed

­

integer

 

Programming

 

Problems

Facility Location Problem – II

The capacity (cost of setting up) of facility

i

∈

{1, 2, …n}

is

α

_i

(

f

_i

)

The production cost in facility

i

is

p

_i

The setup (variable) cost of shipping material from

i

to

j

is

t

_ij

(

c

_ij

)

The capacity of transportation link (

i

→

j

) is

s

_ij

The demand of customer

j

∈

{1, 2, …m}

is

d

_j

Facilities Customers . . . . . . 1 2 n 1 2 m

c

_ij

d

_j

a

_i

, f

_i

∑

∑

∑ ∑

∑

+

+

+

=

j i ij ij j i ij ij i j ij i i i i

X

p

Y

t

Z

c

Y

f

z

, ,

min

Objective:

j

d

Y

_j i ij

≥

∀

∑

Customer demand satisfaction

i

X

a

Y

_{i i} j ij

≤

∀

∑

Facility capacity constraint

(FLII.1)

(FLII.2)

(FLII.3)

Minimize total cost

We define:

X

_i

= 1

if we set up (install) facility i

Y

_ij

= amount shipped from

i

to

j

Z

_ij

= 1

if transportation link (

i

→

j

) is set-up

s

_ij

, t

_ij

j

i

Y

j

i

Z

X

_i

,

_ij

∈

{

0

,

1

}

∀

,

_ij

≥

0

∀

,

j

i

Z

s

Modeling

 

with

 

0

­

1

 

Variables

1. Select at least one item from a setj = {1, 2, …, m} OR operator: y1∨y2 ∨…ym →

∑

≥1

j j

y

2. Select exactly one item

Exclusive OR: y1∨ y2 ∨ … ym →

∑

=1 j

j

y

3. If select k then select l

Implication: k ⇒ l → yl≥yk 4. If and only if Equivalence: k ⇔ l → yk = yl 5. Disjunctions f1(x) ≤ 0 OR f2(x) ≤ 0 y1 + y2 = 1 f1(x) ≤M1(1 - y1) f2(x) ≤M2(1 - y2)

where M1, M2 are sufficiently large numbers

6. Discontinuous Functions/ Domains

Cost = ⎩ ⎨ ⎧ + 0 bx

α

0 = ≤ ≤ x U x L New variable y: If y = 0 ⇒x = 0 If y = 1 ⇒L≤x≤U Formulation: c = αy + bx Ly≤x≤Uy y∈{0,1} α Cost

Modeling

 

with

 

Disjunctions

We often want to enforce different conditions in different cases

Examples: Selection of reactor type

Selection of an additional flash tank

Flash Drum1 Distillation column Water Methanol CO, CO2, H2 Methanol Synthesis Isothermal Reactor Methanol Synthesis Adiabatic Reactor Flash Drum2 3-B) 3-A) _4-A) 4-B) k k k j w x x x k k OUT k IN j unit of parameters l Operationa unit leaving streams of Variables unit feeding streams of Variables stream of variables stream Process

(

)

(

)

⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎣ ⎡ = ≤ = ) ( 0 , , 0 , , k k k k OUT k IN k k k OUT k IN k k S f C x x w g x x w h

Mathematical model of unit k:

(

)

(

)

⎥ ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ = = = = ¬ ∨ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ = ≤ = 0 , 0 0 , 0 ) ( 0 , , 0 , , k OUT k IN k k k k k k k OUT k IN k k k OUT k IN k k k x x C w y S f C x x w g x x w h y

To find the optimal design we want to be able to handle the following logic constraints:

We have to convert the disjunction into MIP constraints, handled by MIP solvers

Reformulations

⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎣ ⎡ ≤ ≤ ≤ ≤ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎣ ⎡ ≤ ≤ ≤ ≤

∨

8 6 10 7 4 2 3 1 2 1 2 2 1 1 x x y x x y

x

₂

x

₁

(

)

(

)

(

)

(

)

(

(

)

)

(

(

)

)

1 1 8 6 1 1 10 7 1 1 4 2 1 1 3 1 1 2 1 2 2 2 2 1 2 1 2 1 1 1 1 = + ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎣ ⎡ − ⋅ + ≤ ≤ + − ⋅ − − ⋅ + ≤ ≤ + − ⋅ − ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎣ ⎡ − ⋅ + ≤ ≤ + − ⋅ − − ⋅ + ≤ ≤ + − ⋅ − y y y M x y M y M x y M y M x y M y M x y M

(

)

(

)

(

)

(

)

⎥⎥_⎦ ⎤ ⎢ ⎢ ⎣ ⎡ − ⋅ + ≤ ≤ + − ⋅ − − ⋅ + ≤ ≤ + − ⋅ − ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎣ ⎡ ⋅ + ≤ ≤ + ⋅ − ⋅ + ≤ ≤ + ⋅ − 2 2 2 2 1 2 2 2 2 2 1 2 1 8 6 1 1 10 7 1 4 2 3 1 y M x y M y M x y M y M x y M y M x y M

(

)

(

)

(

)

(

)

⎥⎥_⎦ ⎤ ⎢ ⎢ ⎣ ⎡ − ⋅ + ≤ ≤ + − ⋅ − − ⋅ + ≤ ≤ + − ⋅ − ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎣ ⎡ ⋅ + ≤ ≤ + ⋅ − ⋅ + ≤ ≤ + ⋅ − 2 2 2 2 1 2 2 2 2 2 1 2 1 7 8 6 1 7 1 7 10 7 1 7 7 4 2 7 7 3 1 7 y x y y x y y x y y x y 7 : M = Selecting

Big

­

M

 

reformulation

Reformulations

⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎣ ⎡ ≤ ≤ ≤ ≤ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎣ ⎡ ≤ ≤ ≤ ≤

∨

8 6 10 7 4 2 3 1 2 1 2 2 1 1 x x y x x y

x

₂

x

₁ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎣ ⎡ ⋅ ≤ ≤ ⋅ ⋅ ≤ ≤ ⋅ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎣ ⎡ ⋅ ≤ ≤ ⋅ ⋅ ≤ ≤ ⋅ = + ⋅ ≤ ⋅ ≤ ⋅ ≤ ⋅ ≤ + = + = 2 2 2 2 2 2 1 2 1 1 2 1 1 1 1 1 2 1 2 2 2 2 2 1 1 1 2 1 1 1 2 2 1 2 2 2 1 1 1 1 8 6 10 7 4 2 3 1 , 1 , y x y y x y y x y y x y y y y M x y M x y M x y M x x x x x x x

(

)

(

)

(

)

(

)

⎥⎥_⎦ ⎤ ⎢ ⎢ ⎣ ⎡ ⋅ ≤ ≤ ⋅ ⋅ ≤ ≤ ⋅ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎣ ⎡ − ⋅ ≤ ≤ − ⋅ − ⋅ ≤ ≤ − ⋅ ⋅ ≤ ⋅ ≤ ⋅ ≤ ⋅ ≤ + = + = 2 2 2 2 2 2 1 2 2 1 2 2 2 1 1 2 2 2 2 2 2 1 1 1 2 1 1 1 2 2 1 2 2 2 1 1 1 1 8 6 10 7 1 4 1 2 1 3 1 1 , y x y y x y y x y y x y y M x y M x y M x y M x x x x x x x

General

 

Mixed

­

integer

 

Programming

 

Model

max

c

T

_x

s.t.

Ax

≤

b

x

_j

integer, j

∈

N

_I

= {1...p}

min

c

T

_{x + d}

T

_y

s.t.

Ax + By

≤

b

x

≥

0

, y integer

ƒ

If

N

_i

=

∅

(i.e. there are no

y

variables)

⇒

Linear Programming (LP) Model

ƒ

If

N

_i

=

N

(i.e. there are no

x

variables)

⇒

(Pure) Integer Programming (IP) Model

ƒ

If

∅ ≠

N

_i

⊂

N

(i.e. we have both

x

and

y

)

⇒

Mixed-Integer Programming (MIP or MILP) Model

A general integer

Y

∈

{0, 1, 2, 3, …N} can be modeled via

k

binary variables

Y

_k

:

∑

=

=

N k k

kY

Y

,... 0 or

So, the general MIP problem is:

min

c

T

_{x + d}

T

_y

s.t. Ax + By

≤

b

x

≥

0

, y

∈

{0, 1}

min

c

T

_{x + d}

T

_y

s.t. (

x, y

)

∈

X

or

LP

­

relaxation

 

of

 

a

 

MIP

 

Model

 

General MIP problem (P):

_min

_c

T

_{x + d}

T

_y

_(P)

s.t. Ax + By

≤

b

x

≥

0,

y

∈

{0,1}

If we replace the integrality requirement

y

∈

{0,1}

with

y

∈

[0,1]

we obtain a relaxation (RP) of (P):

min

c

T

_{x + d}

T

_y

_(RP)

s.t. Ax + By

≤

b

x

≥

0,

y

∈

[0,1]

In general, a relaxation of a problem is obtained by:

ƒ

Either removing constraints (i.e. expanding the feasible region)

ƒ

Decreasing

the objective function (e.g. removing a nonegative term)

min

c

₁T

_x

₊

_c

2T

x

s.t.

A

₁

x

≤

b

₁

A

₂

x

≤

b

₂

x

∈

X

where

x

,

c

₁

,

c

₂

≥

0

min

c

₁T

_x

₊

_c

2T

x

s.t.

A

₁

x

≤

b

₁

x

∈

X

Larger feasible region

(P)

(RP

1

₎

min

c

₁T

_x

s.t.

A

₁

x

≤

b

₁

A

₂

x

≤

b

₂

x

∈

X

Better obj. value at the same point

(RP

2

₎

z(RP2₎ ≤_z(P)

Branch

­

and

­

Bound:

 

Basic

 

Ideas

LP­based method:

ƒ Integrality constraints, y∈{0,1}, are replaced by y∈[0, 1]

max cT_{x + d}T_{y (P) max c}T_{x+ d}T_{y (LRP)}

s.t. Ax + By = b ⇒ s.t. Ax + By = b x≥0, y∈{0, 1}p _x ≥_{0, y}∈_[0, 1]p

ƒ LP problem P(i) is solved at node i, with feasible region F_iand objective Zi

We know how to solve (LRP) effectively using Simplex (vs. other relaxations)

Key observations: 

ƒ If a node is infeasible, then all its descendants will also be infeasible

ƒ A child node will always have a lower (worse) objective function(bounding)

ƒ Any feasible solution Zi_{in node iprovides a lower bound on the optimal solution Z*}

ƒ Easy to add rows (constraints) to (LRP) (why?)

x₁ x₂ 0 1 2 3 1 0 max 3x₁ + 4x₂ 4x₁+3x₂≤10 x₁∈{0,1,2,3}, x₂∈{0,1} x1≤1 x1≥2 (1, 1)-INT ZLP_{= 7.0} (2, 0.66) ZLP_=8.66 (x₁,x₂) = (1.75, 1) ZLP _{= 9.25} x₂≥1 INFEASIBLE x₂≤0 (2.5, 0)-INT ZLP_{= 7.5} Lang & Doig, 1960 x₁≤2 x₁≥3 INFEASIBLE (2,1) ZLP₌₆

Branch

­

and

­

bound:

 

Basic

 

Ingredients

1. The list Lof problems to be solved

2. Relaxation (RP): upper (& lower?) bounding procedure

3. Branching Rule: replace problem P(i) in node iby problems P(i₁), P(i₂), …, P(i_q): FR(i₁)∪…∪FR(i_q) = FR(i) 4. Node (subproblem) selection rule: how to select a problem from the current subproblem list L

5. Pre-processing and variable fixing procedures

ƒ

Method first proposed by Lang and Doig (1960)

ƒ

Around the same time an alternative method was developed (cutting planes)

ƒ

Brute

 

force

branch‐and‐bound remained ineffective for many years

ƒ

Cutting planes were not used for many years

(x₁,x₂) = (0.75, 0.3) ZLP _{= 9.25} x₁≤0 x₁≥1 (1, 0.7) ZLP_=8.75 (0, 0.8) ZLP _{= 8.5} x₂≥1 x₂≤0 (1, 0) -INT ZLP _=7.5 INFEASIBLE On which binary/constraint to branch? x₂≥1 x₂≤0 (0, 0)-INT ZLP _{= 7} Which active node to evaluate next? (0, 1) -INT ZLP _{= 8}

Branch

­

and

­

bound

 

Algorithm

min

c

T

_{x + d}

T

_y

s.t.

(x, y)

∈

X

1. Initialization L = {P} ZU := +∞ 2. Termination If L = ∅ then

If ZU = +∞, then X = ∅ (infeasible problem); STOP

If ZU < +∞, then the solution (x, y)∈X with ZU = cx + dy is optimal; STOP

3. Node Selection and Solution

Using SSR select subproblem (node) P(i)∈L and let L := L\{P(i)} Compute the optimal LP-value Zi and LP-solution (xi, yi) for P(i).

4. Pruning

If Zi≥ZU, then GOTO 2.

(P(i) is either infeasible or dominated by the best solution (upper bound) found so far) If Zi < ZU, then

If yi is integral (i.e. yji is integral ∀j∈NI), then (a better solution is found)

Update upper bound by setting ZU:= Zi

Update list L by removing dominated subproblems: for each i’∈L: If Zi’≥ZU, then L := L\{P(i’)}

GOTO 2.

5. Branching*

Select j∈NI such that yjiis fractional; yj is the branching variable

Update list L by adding programs with fixed yj values

Set L:= L∪ {P(i0), P(i1)} where P(i0) = P(i) ∩ {(x,y)∈ n

+ ℜ × l + ℜ | yj≤ ⎣yji⎦} P(i1) = P(i) ∩ {(x, y)∈ n + ℜ × l + ℜ | yj≥⎡yji⎤} GOTO 2.

Branch

­

and

­

bound

 

Tree

Level

0

1

2

3

4

Nodes

1 = 2

0

2 = 2

1

4 = 2

2

8 = 2

3

16 = 2

4 0 7 8 9 10 11 12 13 14 15 16 17 18 …

Notes:

ƒ

Level denotes the number of fixed variables

ƒ

Total number of nodes =

1 + 2 + … + 2

N

_{= 2}

N+1

_-1,

_where

_N

_{= no of binary variables}

_[Why?]

ƒ

The goal is to explore much fewer nodes via bounding

ƒ

How do we carry out branching?

1 2

Branching

Variable Selection Rules

1. Try to find a good solution fast: Obtain a good lower bound ⇒Prune many nodes fast

2. Choose a variable that results in badsolutions: Prune as many nodes as possible ⇒Keep the b&b tree small

Node selection rules

ƒ Try to find good feasible solutions as early as possible

(x₁,x₂) = (0.75, 0.3) ZLP _{= 9.25} x₁≤0 x₁≥1 (1, 0.7) ZLP_=8.75 (0, 0.8) ZLP _{= 8.5} x₂≥1 x₂≤0 (1, 0) -INT ZLP _=7.5 INFEASIBLE On which binary/constraint to branch? x₂≥1 x₂≤0 (0, 0)-INT ZLP _{= 7} Which active node to evaluate next? (0, 1) -INT ZLP _{= 8}

In general:

ƒ

The number of active nodes increases if the active node is chosen high-up

Variable

 

Selection

 

Rules

We want balanced branching and tight bounds:

choose

j

∈

N

_I

that yields:

max

{

min{ xj=0, xj=1}

}

j Z Z INT ZU₌₁₅ INT ZU₌₁₆ INF FRAC ZU₌₁₈ INF FRAC ZU_=14.5 FRAC ZU₌₁₆ FRAC ZU₌₁₄

INT: Integer feasible

INF: Infeasible FRAC: Fractional

Example

Current best (ZU₎ Closed (infeas or Zi_{> Z}U₎ Open (Zi_{< Z}U₎

Exercise

Variable

 

Selection

 

Rules

We want balanced branching and tight bounds:

choose

j

∈

N

_I

that yields:

max

{

min{ xj=0, xj=1}

}

j Z Z ZU₌₁₅ Current best (ZU₎ Closed (infeas or Zi_{> Z}U₎ Open (Zi_{< Z}U₎

Example

(0,…,0.1, 0.4, 0.95,...1) ZU₌₁₄ ZU_=14.5

Variable

 

Selection

 

Rules

We want balanced branching and tight bounds:

choose

j

∈

N

_I

that yields:

max

{

min{ xj=0, xj=1}

}

j Z Z ZU₌₁₅ Current best (ZU₎ Closed (infeas or Zi_{> Z}U₎ Open (Zi_{< Z}U₎

Example

(0,…,0.1, 0.4, 0.95,...1) ZU₌₁₄ ZU_{=14.1 Z}U₌₁₆ y=0 y=1 ZU_=14.5

Variable

 

Selection

 

Rules

We want balanced branching and tight bounds:

choose

j

∈

N

_I

that yields:

max

{

min{ xj=0, xj=1}

}

j Z Z ZU₌₁₅ Current best (ZU₎ Closed (infeas or Zi_{> Z}U₎ Open (Zi_{< Z}U₎

Example

(0,…,0.1, 0.4, 0.95,...1) ZU₌₁₄ ZU_{=15 Z}U_=15.2 y=0 y=1 ZU_=14.5

Node

 

Selection

 

Rules

9 10

7 8

13.4 13.5

Depth-first Search:

ƒ

Always choose one of the new subproblems just generated; typically the one with the best lower bound

ƒ

Backtrack when node is discarded

5 6 13.2 12.95 z=12 0 12.5 12.8 12.85 13.3 1 2 3 4

Advantages:

ƒ

Can potentially find a (good) solution fast

ƒ

Re-optimization involves the addition of a

single constraint (???)

ƒ

Requires modest storage

Disadvantages:

Node

 

Selection

 

Rules

9 10 13.4 11 12 13.3 13.5 13 14 13.5 13.4 13.1 7 ₈ 13.2 12.95 13.2 13 5 6

Breadth-first Search:

ƒ

Expand all nodes at each level before go to the next level

z=12 0 12.5 12.8 12.85 13.3 1 2 3 4

Disadvantages:

ƒ

Examine many non-promising nodes

ƒ

Feasible solutions are found late

ƒ

Create very large tree (memory requirements)

Node

 

Selection

 

Rules

13.5 11 12 13.1 9 10 13.5 13.4 7 ₈ 13.2 12.95

Best-first Search:

ƒ

Always choose the subproblem with the best lower bound

Advantages:

ƒ

Examines potentially good nodes

ƒ

Uniformly tighten bounds

Disadvantages:

ƒ

Re-optimization is more expensive

ƒ

Tree tends to be larger

13 13.2 5 6 z=12 0 12.5 12.8 12.85 1 2 3 4

Modern commercial solvers use a

combination of depth-first and best-first.

Special

 

Branching

 

Schemes

Special Ordered Sets 1 (SOS1):

x

_i

≥

0

AND

x

₁

+

x

₂

+

x

₃

+

x

₄

+

x

₅

= 1 AND only one non-zero,

x₁+ x₂= 1 x₃+ x₄+ x₅= 1

(0, 0.5, 0.5, 0, 0)

INFEASIBLE

Special Ordered Sets 2 (SOS2):

x

_i

≥

0

AND

x

₁

+ x

₂

+ x

₃

+ x

₄

+ x

₅

= 1

AND at most two non-zeros AND consecutive

x₁=0, x₂=0 x₃=0, x₄=0 (0, 0.5, 0, 0.5, 0) INFEASIBLE x₁= 1 (0, 0.5, 0.5, 0, 0) INFEASIBLE x₂= 1 x₃= 1 x₄= 1 x₅= 1

or

Pre

­

processing

ƒ

Preprocessing applies simple logic to reformulate and tighten the LP relaxation

ƒ

For MIP problems is better to spend extra time initially to reduce the possibility of long runs

Generate Bounds

ƒ

e.g., If the upper bound of a 0-1 variable is less than 1, then it can be fixed to 0.

Detect Infeasibilities and Redundant Constraints

ƒ

If

L

_i

≤

Ax

i

≤

_U

i

(for

l

≤

x

≤

u

) then

Ax

i

≤

b

i

is redundant if

U

i

≤

b

i

and is infeasible for

L

i

< b

i

.

Probing

Temporarily set a 0-1 variable to 0 or 1 and then redo the logical testing.

ƒ

5

x

₁

+

3

x

₂

≥

4

becomes infeasible when

x

₁

=0

⇒

x

₁

=1

in every feasible solution

ƒ

5

x

₁

+

4

x

₂

+ x

₃

≤

8

If

x

₁

=

1 then

x

₂

=

0 and vice versa

⇒

x

₁

+ x

₂

≤

1

ƒ

2

x

₁

+ x

₂

+ x

₃

≥

1

becomes strictly redundant when

x

₁

=

1

.

⇒

Can be replaced by

x

₁

+ x

₂

+ x

₃

≥

1

(1, 0.75, 0)

(0.5, 0, 0)

Pre

­

processing

In summary preprocessing:

ƒ

Improves bounds

ƒ

Identifies infeasibility and redundant constraints

ƒ

Fixes variables

ƒ

Generates new valid inequalities

ƒ

Tightens existing inequalities (coefficient improvement)

Not possible to perform all operations and save all information

Computational Effort for preprocessing:

1. Techniques applied to one row

⇒

Always worth doing

2. Saving and preprocessing implication and clique tables

3. Full probing on the matrix

Common strategy:

1. Generate clique and implication inequalities

2. Store them in separate tables

Heuristics

ƒ

Solution Time proportional to tree size

ƒ

Tree is large when there are many active nodes; i.e. nodes with

z

i

_<

_z

best

_{= Z}

U

ƒ

How do we keep the size of the tree small?

Try to reduce

z

i

_{as fast as possible}

_⇒

_{Variable Selection Rule}

Try to find large

z

best

_{as fast as possible}

_⇒

_Heuristics

Also for large scale instances: “good feasible solutions may be all we can hope for”

If we know the structure of the problem: Use any known heuristic to provide an initial

z

best

e.g. use Lin-Kernigham heuristic to get a usually very good solution for the TSP

If we do not know a heuristic: Use LP-based heuristics:

ƒ

Enumerate 0-1 vectors in the neighborhood of the current LP solution and check feasibility

OCTANE heuristic (Balas et al., 2002)

ƒ

Successive rounding heuristics

Fractional variables sequentially rounded until an integral solution or infeasibility is detected

Equivalent to “diving” in the b&b tree: branching on a variable and examine only one child

Motivation

 

for

 

Branch

­

and

­

Cut

x₁ x₂ 0 1 2 3 1 0 max 3x₁+ 4x₂ (M) 4x₁+3x₂≤10 x₁∈{0,1,2,3}, x₂∈{0,1}

IP Feasible Region X

x₁ x₂ 0 1 2 3 1 0

LP Relaxation Feasible Region

x₁ x₂

0 1 2 3

1

0

Convex Hull of IP Feasible Region

max 3x₁+ 4x₂ (M) 4x₁+3x₂≤10 x₁∈{0,1,2,3}, x₂∈{0,1} max 3x₁+ 4x₂ (M) 4x₁+3x₂≤10 x₁∈{0,1,2,3}, x₂∈{0,1}

Valid

 

Inequalities

 

and

 

Cutting

 

Planes

x₁ x₂ 0 1 2 3 1 0

LP Relaxation Feasible Region

Facet Valid Inequality Cutting Plane x₁ x₂ 0 1 2 3 1 0

4

x

₁

+

3

x

₂

≤

10

⇒

x

₁

≤

2

ƒ

If we have

CH(X)

we solve just one LP

ƒ

We need facet defining inequalities

ƒ

Has been done for some problems

ƒ

In general very difficult

ƒ

Try to approximate instead