LECTURE: INTRO TO LINEAR PROGRAMMING AND THE SIMPLEX METHOD, KEVIN ROSS
MARCH 31, 2005
DAVID L. BERNICK [email protected]
1. Overview
• Typical Linear Programming problems
• Standard form and converting to Standard form • Geometry of Linear Programming
• Extreme points, linear independence and bases • Optimality conditions
• The Simplex method
2. Typical Linear programming Problems 2.1. Product Mix problem
Problem: How much beer and ale should be produced?
available resources = amt. item(pounds) 480 corn 160 hops 1190 malt (1) required resources =
item corn hops malt
(barrel) (pounds) (ounces) (pounds)
Ale 5 4 35 Beer 15 4 20 (2) prof it = · Ale $13/barrel Beer $23/barrel ¸ (3) 1
Analysis:
A = Ale(barrels) B = Beer(barrels) Objective f unction
M ax{P rof it = 13A + 23B} Constraints : Subject to(s.t) 5A + 15B ≤ 480 4A + 4B ≤ 160 35A + 20B ≤ 1190 A, B ≥ 0 In general: ci= producti prof it xi= producti count bj = resourcej availability
ai,j = resourcej required f or producti
objective f unction = max{ X
i∈{products} cixi} s.t. X i∈{products} ai,jxi≤ bj xi≥ 0 2.2. Transportation Problem Problem:
• Production of computers in Singapore and Oakland • Distribution centers in Oakland, Hong Kong and Istanbul
Supply, demand and cost summary:
Supply, demand, cost =
ship.cost
Oakland HongKong Istanbul Supply
Singapore 85 37 119 500 Hohboken 53 189 94 300 Demand 350 250 200 (4) Objective: meet demand with minimum total cost.
Analysis:
ci,j = ship.cost f rom i to j
Si = supply f rom sitei
Dj = demand f rom sitej
i ∈ {Singapore, Hohboken}
j ∈ {Oakland, HongKong, Istanbul} xi,j = count shipped f rom i to j
objective =X i,j ci,jxi,j s.t.X j xi,j = Si∀i(supply) X i xi,j = Dj∀j(demand) xi ≥ 0 3. Other LP examples • Blending Problem • Diet Problem • Assignment Problem 3.1. Blending Problem
Problem: Consider a constraint where at least 10% of output must be from 1 variable.
We then have a constraint of the form: Pxi
ixi ≥ 0.10. This is not a linear equation but we can change it to: xi ≥ 0.10 ∗
P
ixi and now it is a linear form.
4. Key elements of a LP
• Proportionality - the function depends on a proportion assigned to
a variable.
• Additivity • Divisibility
4.1. Steps in building a LP
• Identify the activities. • Identify the items.
• Identify the Input / Output coefficients. • Write the constraints.
• Identify the coefficients of the objective function.
5. Geometry of a LP
Consider the plot of the solution space for the following:
objective f unction : max{x1+ x2} s.t. 3x1− 5x2 ≤ 15
3x1− 5x2 ≤ 12 x1, x2 ≥ 0
As a quick way to find the regions specificd by the inequality, consider weather (0,0) is a solution for the inequality or not. In this case, (0,0) is part of the solution spacec for all constraints.
Geometrically, this problem can be viewed as Fig:??. As can be seen, the solution is at a corner. This is true in general for Linear Programming problems - one of the optimal points will be at a corner. This is the basis of the Simplex method.
6. Standard Form Types of LP descriptors: Objective f unction : max(c1x1+ c2x2+ . . . cNxN s.t a1,1x1+ a1,2x2+ . . . + a1,NxN = b1 a2,1x1+ a2,2x2+ . . . + a2,NxN = b2 xj ≥ 0, j = 1, N
Figure 1. Graph of feasible region, bounded by all constraints, with family of curves of x1+ x2 and max(x1+ x2). or consisely: max(c0x) s.t. Ax = b x ≥ 0
where A is an [m x n] matrix of N variables and m constraints.
All LP problems can be converted to standard form. The following situations can arise, and are converted to standard form as follows:
• inequalities • free variables • MINimizations
6.1. Handling Inequalities
For example, the constraint x1+ x2 ≤ 4 can be handled by introducing
a new, surplus variable x3. This constraint then becomes x1+ x2+ x3 =
6.2. Handling Free variables
In the case where a variable is free (x1 is unconstrained), then we once
again introduce a new, surplus variable. For example:
x1 ∈ <(f ree variable) objective f unction : min(x1+ x2) s.t. x2≥ 0 x1+ x2= 3 replace x1= x+1 − x−1 x+1, x−1 ≥ 0
rewritten in standard form:
objective f unction :
min x1+ x2 new variable does not appear in the objective function s.t. x2 ≥ 0
x+
1 + x−1 + x2 = 3 x+
1, x−1 ≥ 0
6.3. Handling MIN objectives (vs. MAX)
To convert a MIN to a MAX, negate the terms. For example: min x1+ 3x2= max −x1− 3x2
. Remember, any surplus variable that is introduced does not appear in the objective function - only in constraints.
7. Solutions, Extreme points and Basis
• How many solutions are there to a set of linear equations? • Convexity of a feasible region.
7.1. How many solutions exist for a set of linear equations How many linearly independent rows and columns exist in the set? In matrix notation, the set of equations is described by
Ax = b
. If A is a square matrix (NxN) and det(A) 6= 0 then x = Ab is unique. In general, A is rectangular (m x N), with many variable and few con-straints.
7.2. Convexity of a feasible region
X ≡ a convex set iff ∀x1, x2 ∈ X
λx1+ (1 − λ)x2 ∈ X , {0 ≤ λ ≤ 1}
or, for any two points in set X , any point between any point between those 2 points is also in the set.
7.3. Extreme Point of X ¯
x is an extreme point iff
f or distinct x1, x2 ∈ X
if, ¯x = λx1+ (1 − λ)x2, {0 ≤ λ ≤ 1}
=⇒ λ ∈ {0, 1}
then x is an extreme point¯
or, if ¯x is not between 2 other points in the set, then it is an extreme point.
8. Linear independence of vectors
• Basis of a matrix
• A basic solution of an LP
• Basic Feasible solution (Corner Point Feasible)
8.1. Basic Feasible solution(bfs)
~x is an extreme point of a solution space iff it is a bfs of Ax = b, {x ≥ 0}.
Key fact: If an LP has an optimal solution, it exists at a corner of the feasible region.
8.2. Basis of a matrix
• Linear independence
Let V1, V2, . . . , VN be vectors. They are linearly independent,
if : α1V1+ α2V2. . . + αNVN = 0 =⇒ α1= α2= . . . αN = 0
Basis of matrix A is a maximal linearly independent set of columns of A. For example: A = · 1 1 0 1 1 1 ¸ (constraints)
Basis(A) = 1st and 3rd columns of A, or 2nd and 3rd columns
Only a certain number of constraints need to be binding.
8.3. Rank of a Matrix
Rank of a matrix = number of independent columns in the matrix. [m x n] matrix A is full rank if rank(A)=m. Or, if there are at least as many idependent columns as there are constraints.
A = a1,1 a1,2 . . . a1, N .. . am,1am,2 . . . am,n
8.4. Linear Programming
Write A = [B, N ] where B is a basis of A.
A = · 1 2 0 1 1 0 0 1 1 1 ¸ Basis of A = B = · 1 0 0 1 ¸
Lef tover or N ull = N =
· 2 1 1 0 1 1 ¸ max cTx s.t. Ax = b x ≥ 0 A = [B, N ]
Rewrite constraints as:
[B, N ] µ xB xN ¶ = b set xN = 0 xB = B−1b BxB+ N xN = b Set N xN = 0 XB = b XB = B−1b
9. Simplex Method - Overview
• Checks Corner Points
• Looks for better solutions at each iteration
Simplex Algorithm:
• Find a starting point - a corner. • Test this point for optimality
• Stop if this point is optimal, otherwise repeat
One basic variable is replaced by another. Optimality test identifies the basic variable to replace.
