Basic Feasible Solutions
4.1 Geometry and Basic Feasible Solutions
In the last few lectures, we developed geometric intuition that allowed us to find the optimal solution of linear programs with 2 and 3 variables. Then, we moved to higher dimensions by introducing definitions for extreme points. We saw that one of these extreme points will always be an optimal solution to a linear program, provided that the program is not infeasible or unbounded, and gave an algebraic characterisation of extreme points by introducing the notion of a basic feasible solution. At this point, we have reduced the problem of solving a linear program to the problem of finding the best basic feasible solution of that program. Before going further, let’s see an example bringing together some of these principles.
Let’s consider the following linear program in 2 variables:
maximise 4x1+1 2x2
subject to x1 + x2 ≤ 3 1
2x1 + x2 ≤ 2
−1/2x1 + x2 ≥ −1 x1, x2 ≥ 0
Notice that the last constraint that is a ≥ constraint instead of a ≤. Before sketching, we should always convert all of our constraints to ≤ constraints, so we multiply both sides by −1. We get the following equivalent linear program in standard inequality form:
maximise 4x1+1 2x2 subject to x1+ x2 ≤ 3
1
2x1+ x2 ≤ 2 1
2x1− x2 ≤ 1 x1, x2 ≥ 0
(4.2)
We want to see what the feasible region looks like, so we plot the boundary of each constraint. That is, we plot the lines we get when we change the ≤ signs into = signs. We get the following:
x1 + x2 =
3
21x1 +x2
=2
1
2x1− x2
=1
x1 x2
Which part of the picture corresponds to the feasible region? We need to figure out which side of each line the feasible region lies on. One easy way is to pick any point we want on each line and then draw the normal vector for our line.
For example, line 12x1 + x2 = 2 is a linear equation of the form aTx = 2, where aT= (12, 1) and x = (x1, x2). So, we pick any point on this line and draw a vector starting at the point and moving 1/2 unit to right (corresponding to the 1/2 in the first coordinate of a) and 1 up (corresponding to the 1 in the second coordinate of a). Intuitively, the vector we have just drawn indicates the direction in (x1, x2) plane that corresponds to making the expression aTx = 12x1+ x2 larger.
Our line shows us the values for which this expression is equal to 2. Because we want to also include values that are smaller than 2, our feasible solutions must all lie on the opposite side of our normal vector a.
If we repeat the same process for each different line, and consider the intersec-tion of all these regions, together with the non-negativity restricintersec-tions on x1 and x2, we get a picture that looks like this:
x1 + x2 =
3
21x1 +x2
=2
1
2x1− x2
=1
x1 x2
Normally, we would finish by sketching the direction c corresponding to the ob-jective function (here, cT= (4, 1/2)) and use it to solve our program. Instead, let’s see how this might be done using calculation and our notion of basic feasible solutions. We saw in the last lecture that any linear program that has an optimal solution must have an optimal solution that is also a basic feasible solution. But, what do these basic feasible solutions look like? We saw that they correspond to extreme points, which are like corners of our feasible region. Let’s see in more detail exactly how this correspondence works.
In order to talk about basic feasible solutions, we need to rewrite our program into standard equation form. We have 3 inequalities in 4.2. We introduce a new slack variable for each of them to get the following program:
maximise 4x1+ 1 2x2
subject to x1 + x2+ s1 = 3 1
2x1 + x2+ s2 = 2 1
2x1− x2+ s3 = 1 x1, x2, s1, s2, s3 ≥ 0
(4.3)
We used to have n = 2 variables and m = 3 constraints. Now, we have n + m = 5 variables: 2 original variables plus one new slack variable corresponding to each constraint. We can write our system of equations more succinctly in matrix form
as:
Remember that in a basic feasible solution, we require that all the variables that take non-zero values correspond to a linearly independent set of columns in the matrix above. Since our matrix has m = 3 rows, at most m = 3 columns can be linearly independent, and we should expect that any basic feasible solution will select at most 3 variables to be set to non-zero values, or, equivalently, selecting 3 linearly independent columns of our matrix. Suppose that we select columns 1, 2, and 5 (note that these are indeed linearly independent). Then, in a corresponding basic feasible solution we are allowing x1, x2 and s3 to take non-zero values, and requiring that s1 and s2 be set to zero. We say that x1, x2, and s3 are the basic variables in this solution, and that s1 and s2 are the non-basic variables.
Let’s now see what the effect of this choice is. Since we have s1 = 0, then the first equation in 4.3 reads:
x1+ x2 = 3
This is exactly the same as saying that the constraint corresponding to slack variable s1 is tight in our original program (4.2). Similarly, since we have s2 = 0, then we must have
1
2x1+ x2 = 2
and so our second constraint in (4.2) is tight. Consider the following picture, in which we have drawn the equations corresponding to each of these tight constraints:
1
2x1− x2
=1 x1 +
x2 = 3
21x1 +x2
=2
x1 x2
If set s1 and s2 to zero and write out the system of constraints for our standard equation form program (4.3), we get:
x1+ x2 = 3 1
2x1+ x2 = 2 1
2x1− x2+ s3 = 1
We find that this set of equations has a unique solution, namely x1 = 2, x2 = 1, s3 = 1. This, then is our basic feasible solution, which was obtained by selecting a linearly independent set of columns 1,2, and 5 and then letting x1, x2, s3 be our basic variables. If we plot just the x1 and x2 values for this solution on our picture, we get exactly the intersection point of these 2 red lines, which is also an extreme point of the feasible region. We showed in the last lecture that this will always happen—basic feasible solutions correspond exactly to extreme points of our feasible region.
So, we see that whenever a slack variable in our standard equation form pro-gram (4.3) is non-basic (that is, set to zero), it means that its corresponding constraint in our original program (4.2) is tight, and that this means that x1 and x2 must lie on this constraint’s boundary line. What if one of x1 or x2 is set to zero? Remember that we actually have 2 extra boundaries in our feasible region, corresponding to the restrictions x1 ≥ 0 and x2 ≥ 0 in both of our programs. If, for example x1 = 0, then one of these is tight. Indeed, suppose that we choose
columns 2,4, and 5 in our matrix to be basic. Then, in any corresponding basic feasible solution, we must have x1 = 0 and s2 = 0 (so 12x1+ x2 = 2). If we draw these tight restrictions/constraints, we see:
x1 + x2 =
3
12x1− x2
=1
1 2x1
+x2
=2
x1=0
x1 x2
Again, we see that this gives us an extreme point of our feasible region. If we set x1 and s1 to zero our constraint equations become:
x2+ s1 = 3 x2 = 2
−x2+ s3 = 1
which has a unique solution x2 = 2, s1 = 1, s3 = 1. Looking at the variables x1 and x2 from the original problem (4.2), we find x1 = 0, x2 = 2. So, again, our basic feasible solution lies at the intersection of 2 lines, which is an extreme point of the feasible region.
As a last example, let’s suppose we choose columns 1, 4 and 5 of our matrix.
This will give us basic variables x1, s2, s3 and non-basic variables x2, s1. As before, we will have 1 tight constraint, giving us x1 + x2 = 3, and one tight restriction, giving us x2 = 0. When we draw these get:
21x1 +x2
=2
1
2x1− x2
=1 x1 +
x2 = 3
x2 = 0
x1 x2
Now, we find that our intersection point is strictly outside the feasible region. How could this be? Let’s check our equations. If we set x2 and s1 to zero, we get:
x1 = 3 1
2x1+ s2 = 2 1
2x1+ s3 = 1
These equations have a unique solution x1 = 3, s2 = 12, s3 = −12. Indeed, if we check, the point x1 = 3, x2 = 0 corresponds to the intersection in our picture.
However, notice that we can tell right away that this solution is not feasible, since it sets s3 = −1, but (4.3) required that all variables were non-negative. The constraint in (4.2) that corresponds to the slack variable s3 is 12x1 − x2 ≤ 1, and this is exactly the constraint that we are on the wrong side of!
In summary: Suppose we write a linear program with m inequality constraints and n variables in standard equation form with a matrix A that has m + n columns (n for the original variables, and m for the slack variables). Then any m linearly independent columns of A will correspond to a possible set of basic variables.
We set the other non-basic variables to zero, which will correspond to n tight constraints/restrictions in our original program. You can show that any way of doing this will always result in a single, unique solution for the values of the basic variables (see this week’s homework), and so in fact the n tight constraints
corresponding to a basic feasible solution will always be linearly independent. If this solution sets all of the basic variables to be non-negative, then it is a basic feasible solution.