of eq. (S). We can modify the constraints by adding an artificial variable w, where w ∈ R, to the constraints. Consider the phase-I problem:
(phase-I) min
k
X
i=1
wi s.t. Ax + w = b, x ≥ 0, w ≥ 0. (3.17)
With an initial basic feasible point [x, w]> = [0, b]>, we can solve eq. (3.17) by the simplex method and obtain an optimal solution [x∗, w∗]. If w∗ = 0, then we go to phase II which is to solve the original problem with x∗ as a feasible basic solution to start with. Otherwise, the original problem has no feasible solution [7, p. 39].
3.4
Bland’s rule
When the simplex method solves degenerate problems, the method may repeat the same sequence of degenerate pivots associated with a non-optimal solution. In this case, the method is said to be cycling and will never terminate [7,§3.5].
To avoid cycling, we can apply Bland’s rule to choose the entering and the leaving variables to a basis. The idea of Bland’s rule is to determine the choice of both the entering and the leaving variables. To do so, we first order the variables x1, x2, . . . , xn. At each iteration, among all non-basic variables with negative reduced
cost, the simplex method selects the entering variable with the smallest index to enter the basis. Next, Bland’s rule breaks a tie in the ratio test by choosing the basic variable with the smallest index from all potential leaving variables to leave the basis [7,§3.6].
3.5. Summary 28
3.5
Summary
To conclude, we studied the simplex algorithm and the two-phase method to obtain initial basic feasible solutions. For an implementation, we used the revised simplex method as it needs less storage and less computation. We also discussed how the simplex method can avoid cycling when solving degenerate problems by applying Bland’s rule.
For the next two chapters, we will look at two interior point methods: the affine scaling and the primal-dual methods. Each of them is currently considered to be an efficient general method for solving linear programs.
Chapter 4
Affine scaling methods
The affine scaling method was first introduced by Dikin in 1967 [10, p.336]. Unfortu- nately, his work was not widely well-known until several researchers independently rediscovered that the affine scaling method is a simple version of Karmarkar’s algo- rithm [44].
In this chapter, we will describe the affine scaling method and its implementation in sections 4.1 and 4.2. Similar to the simplex method, the affine scaling needs an interior feasible solution to start with, therefore in section 4.3, we will explain how to find starting points.
4.1
Formulating affine scaling methods
The affine scaling method solves the linear program in the form of (P), that is,
min c>x subject to Ax = b and x ≥ 0. (P) The idea of the method is to apply the affine scaling transformation to keep it- erative points away from the boundary so that it has a decent improvement. At each iteration, the method generates an interior feasible solution that decreases the
4.1. Formulating affine scaling methods 30 corresponding objective function value.
Let k be the number of iteration. Suppose that we have a strictly positive initial solution xk which is not optimal, so we must find another solution denoted by
xk+1= xk+ βd, (4.1) where β > 0 is a step length and d is a direction which decreases the objective function value. Note that a direction d = −c is a good choice as it makes the objective function value decrease. To stay in the feasible region, the new point must satisfy
Axk+1= A(xk+ βd) = Axk+ βAd = b. (4.2) Therefore, the direction d must satisfy Ad = 0 which is equivalent to projecting d onto the null space of the matrix A [10, p.336]. Since A has full rank, the projection matrix
PA = I − A>(AA>)−1A (4.3)
maps any direction d onto the null space of A [44, p.348]. Then, we project the proposed direction −c onto the null space of A and obtain a direction:
d = −PAc. (4.4)
Note that PA>= PA and PA2 = PA, so
c>d = −c>PAc = −c>PAPAc = −(PAc)>(PAc). (4.5)
Therefore, d = −PAc also yields a decrease of the objective function value [44,
4.1. Formulating affine scaling methods 31 If PAc = 0, then for any direction γ,
c>xk+1 = c>(xk+ βPAγ) = c>xk+ β(PAc)>γ = c>xk. (4.6)
In this case, the objective function value remains the same. Therefore, xk is an
optimal solution [7, Lemma 7.2, p.150].
Note that if a component of xk is closer to the boundary, then the method can take only a small step without making xkcross zero. Consequently, an improvement in the next iteration is very small. In fact, this issue can be avoided if xk is near the centre of the feasible region. Therefore, we should scale xk so that we can make
a decently long step for the next iteration. We also scale the linear programming problem and compute the scaled direction.
We first formulate a scaled linear programming problem. The following part follows [10, §10.5]. Let xk > 0 be an interior feasible point of (P). We scale each component by mapping each of them to unity so that each of them stays at least one unit away from zero. Let X = diag(xk) be the n × n diagonal matrix whose diagonal
elements are the components of xk. Note that X> = X. As xk is an interior point,
the diagonal elements of matrix X are strictly positive and therefore, X is invertible. Since Ax = AXX−1x and c>x = c>XX−1x, if we define
z = X−1x, (4.7)
then we have zk = X−1xk = e where e is a vector all of whose elements are equal
one. We must scale A and c as: ˜
4.1. Formulating affine scaling methods 32 Taking z, ˜A and ˜c back to (P), we obtain the scaled linear programming problem:
min ˜c>z subject to Az = b˜ and z ≥ 0. ( ˜P ) Note that x is feasible in the (P) problem if and only if z is feasible in the ( ˜P ) problem and they have the same objective function value. With a step length β > 0, we move from a current scaled solution zk to a new scaled solution:
zk+1 = zk+ β ˜ d
|| ˜d|| (4.9) where ˜d = −PA˜˜c is the direction of the next scaled iterate and ||v|| = (Pni=1vi2)
1 2.
Note that we divide ˜d by || ˜d|| in eq. (4.9) to normalise the direction ˜d [33, p.118]. Again, if PA˜c = 0, then by the same argument as in eq. (4.6), we achieve the optimal˜
value. Finally, we transform the scaled solution zk+1 back to the original variable
[33, p.118]: xk+1 = X zk+ β ˜ d || ˜d|| ! = xk+ βX ˜ d || ˜d||. (4.10) For any β > 0, we have
c|xk+1= c|xk+ c| βX ˜ d || ˜d|| ! (by eq. (4.10)) (4.11) = c|xk− β || ˜d||(˜c |P ˜ A˜c) by ˜c = Xc, ˜d = −PA˜c˜ (4.12) = c|xk− β || ˜d||(PA˜˜c) |(P ˜ A˜c) by P 2 ˜ A= PA˜ (4.13) = c|xk− β || ˜d|| ˜ d|d˜ by ˜d = −PA˜c˜ . (4.14)
Therefore, we can detect unboundedness from the sign of ˜d.
Lemma 14. [7, p.150] If ˜d ≥ 0 and ˜d 6= 0, then the problem (P) is unbounded. Proof. Suppose ˜d ≥ 0 and ˜d 6= 0, then for any β > 0, it is seen from eq. (4.14)
4.2. Implementation 33