Integer Programming: NAZ cut and A-T
cut
Sanam Haseen
Sheema Sadia
Abdul Bari
Qazi Mazhar Ali
Department of Statistics and Operations Research, Aligarh Muslim University,
Aligarh, 202002 India.
Abstract
This paper gives a brief knowledge of the different methods used for solving Integer programming problem along with comprehensive literature review of the work done in this field till date. We know that there is a need of integer solution in various fields of work like industries, production units etc. So there is a demand to develop some fully fledged methods for solving integer programming problem. The techniques developed so far for solving IPP are broadly divided into two categories, cutting plane methods and branch and bound methods. Some important and widely used methods under these are discussed in this paper, along with a bibliographic review of references directed towards IPP. At the end a study of NAZ cut and A-T cut is also done.
Keywords: Branch and Bound; Cutting Plane Methods; NAZ cut; A-T cut.
1. Introduction
We encounter many real life problems where the solution which is needed to be an integral value, and non-integral solutions are meaningless for the problems. For example in problems like distribution of goods, production, scheduling, machine sequencing etc. the solution is required to be an integer value. Few more include planning problems such as capital budgeting, facility location, portfolio analysis and design problems such as communication and transport network design, circuit design and the design of automated production systems. In such cases, mathematical programming problem (MPP)with integer restrictions is considered. Thus an MPP with integer restrictions is termed as “Integer programming problem" (IPP) and the mathematical formulation of IPP is given as
maxz=g0(x1,x2,...,xn) (1) m i
b or or x x x
gi( 1, 2,..., n) = i, =1,2,, (2)
. } {1,2,..., 0,
integer x
n N
j x
j
j
(3)
If all xjs are integer, i.e. all the variables xj are restricted to have integer values, the problem is called as a pure integer programming problem. Otherwise, if some and not all xjs are integer then it is a mixed integer programming problem. Here, Equation 1 is the objective function, which can be a linear or non-linear function. Equation 2 and 3 are known as the constraints with non-negative restrictions.
Bendre’s[6] partitioning scheme was devised by Harris[19] for mixed-integer programming. Trotter and Shetty[29] proposed an algorithm for the bounded variable pure integer programming problems. Granot and Granot[17] constructed a new cutting plane algorithm for solving the integer fractional programming and mixed integer fractional programming problems using Charnes and Cooper[10] approach for solving the integer continuous fractional programs.
Yet, another approach for solving integer programming problem is the branch and bound method. Branch and bound method is an enumeration technique which involves setting up a tree of linear programming problems. The root of the tree is the problem in which all integrality constraints are ignored which eventually branch out to attain integer optimal solutions. The constraints are thought to be as branches of the tree. One can then form sub-branches on the values of other variables that must be integers. The problem is solved when one has a feasible optimal solution on one branch of the tree, and lower values of decision variables on all other branches, whether feasible or not. Land and Doig[22] was the first to propose this approach. Bertier and Roy[7], later, presented a general theory for branching and bounding. Balas[1] restated their theory in far more simpler form which Mitten[26] generalized and extended slightly.
A lot more work has been done on integer and mixed integer programming problems. Few of them are due to Guignard and Spielberg[18], Lenstra[24], Hoffman and Padberg[20], Savelsberg[28], Balas et al.[2] and Wiesmantel[31]. Recent works of this century are due to Kesavana et al.[21], Frangioni and and Gentile[13], Vyve and Wolsey[30], Patel and Chineck[27], Lay and Jacobson[23], Cadoux[9].
2. Need for Integer Programming Problem
Now, take into consideration a Linear Programming Problem (LPP). If someone wants to get an integer solution to the given LPP, an obvious idea that comes into mind is to round off the continuous optimal solution obtained.
This approach is sometimes useful where the variables are very large. But in most of the cases the approach has a major drawback.
One is that after rounding off, an optimal solution may not any longer remain a feasible solution. For instance, consider this example
.
0
0, 2
1 3
2 1 2 = max
2 1
2 1
2 1
2 1
2 1
integers x
and x
x x
x x
x x to subject
x x z
(4)
The optimal solution for the L.P. relaxation is
2 1 1 = 2,
= 2
1 x
x , but it is impossible to round off the noninteger variable to a solution that retains feasibility. See figure 1.
In yet another case, it is very likely to happen that the solution obtained after rounding off may remain feasible but is far from the optimal solution. For example in this IPP
.
0 0,
6.5
15 3
10 2
2 4 = max
2 1
2 1
2 1
2 1
2 1
2 1
integers x
and x
x x
x x
x x
x x to subject
x x z
(5)
The optimal solution to the problem is far away from the nearest rounded off integer solution that happen to be feasible. See figure 2.
Figure 2. Nearest integer solution is far away from the optimal integer solution.
3. Different methods to solve IPP
So, from such examples as given above we can conclude that rounding off the optimal solution is not always a useful technique to get an integer optimal solution. Thus, to solve LPP with integer restrictions well defined methods started to develop. Although a large number of methods have been developed till now. But among those, two methods are mainly used.
1. Branch and Bound methods 2. Cutting Plane methods 3.1 Branch and Bound Methods
Branch and Bound is a widely used method for finding an optimal solution for an IPP. This approach basically forms a tree of linear programming models whose root is free from all integrality constraints and branches grow to an optimal integer solution of the initial model.
The application of Branch and Bound Method involves basically three operations: Branching, Bounding and Fathoming.
Branching is the process in which the continuous solution space is divided into continuous disjoint subspace (sub problems) which are also continuous.It is done to eliminate the part of the continuous solution space that are not feasible for integer problem. By branching the solution space reduces at every stage making it easy to search the integer solution.
Fathoming is a process of terminating a problem from further calculations. In Branch and Bound Method each created sub problem is solved as a continuous problem. A fathomed sub problem is the one that has been discarded for further consideration of branching. There may be three reasons for fathoming a sub problem.
1. The sub problem yields an integer solution. This provides a lower bound on the optimum integer solution. 2. The continuous optimal solution of the sub problem is no better than the current best integer solution. 3. The sub problem is infeasible.
The three operations of branch and bound leads to an optimal solution that is eventually determined following an optimality criterion.
Optimality Criterion In branch and bound the optimum integer solution surfaces when a sub problem having the largest upper bound among all sub problems yields an integer solution.
Land and Doig[22] were the first to develop branch and bound method. In the Land and Doig algorithm it can be seen that the problem becomes complicated due to large number of branches that could originate from the same node. The confusing and tedious work of branching out was streamlined by the following branching rules given by Dakin[11].
Let xk,xkI whose non-integer value at a given node is x*k. Then a necessary condition for xk to be an integer is that
xk [x*k] or xk[xk*]1
This means that the range [xk*]xk[xk*]1 is not feasible with respect to the integer problems.
Thus at each node only two branches are needed, namely one corresponding to xk [x*k] and the other corresponding to xk [x*k]1. As these additional constraints are inequalities the entire range of possible integer points is covered by the two branches and no further branching from the same node would be necessary. The advantage of Dakin’s method over Land and Doig method of branch and bound can be easily concluded through this example
.
0 0,
45 9 5
6
8 5 = max
2 1
2 1
2 1
2 1
2 1
integers x
and x
x x
x x
x x to subject
x x z
Figure 4. Branch and bound tree by Dakin’s method.
Although, branch and bound method is very useful technique to solve IPP, there are some drawbacks, like, the problem becomes complicated for large number of decision variables.
3.2. Cutting Plane Methods
The convex hull of a set of points is the smallest convex set containing those points. The optimal solution of the IPP could be determined by solving a single LPP in which the convex hull is used as the feasible region. This is because the extreme points of the convex hull corresponds to integer solution. This is the basic motivation behind the cutting plane methods.
In real, it is very difficult to actually compute the convex hull of a set of feasible integer point. Thus, the target of a cutting plane method is to construct such a convex hull such that it is in the vicinity of optimal integer solution. This is done in a systematic manner by introducing additional constraints that cuts off portion of the solution space that are not integer feasible. The purpose of the this cutting is to get an optimal integer solution on the extreme point of the developed feasible region. Once this is accomplished, the optimal integer solution can be found by solving corresponding LP relaxation.
3.2.1. Dantzig’s Cut
Dantzig et al.[12] observed that a legitimate cut for a given non-integer optimal solution to an integer program is that the sum of the non-basic variables must equal or exceed one i.e. xj1 for some jNB. An obvious justification is that under no circumstance could the sum of non-basic variables be negative. Thus a constraint which must be satisfied by every integer solution is
1
j NB j
x
(6)
After introducing s0 as slack variable, equation 6 can be expressed as:
1 = s xj NB j
0 1
=
j NB j
x s
or
(7)
Equation 7 is the Dantzig’s Cut.
The procedure introduced by Dantzig can be rewritten into the form of steps as below
3. If some of the variables are not integers add ‘Dantzig cut’ at the end of the last row of the simplex tableau. 4. Solve the altered problem using dual simplex method.Go to Step 2.
However, as shown by Gomory[14] and Hoffman[20], using Dantzig’s cut the convergence of the procedure is not guaranteed, that is, it is not necessary that in a finite number of steps the optimum integer solution is reached. Later on Bowman and Nemhauser[8] have shown that a slight modification in Dantzig’s cut given in 7 can provide a finite algorithm. Ralph Gomory has made many outstanding contributions to the field of IPP. He developed some excellent cuts among which are :
3.2.2. Gomory’s Fractional Cut ([14])
Gomory’s fractional algorithm may be described in the following steps: 1. Solve the LPP using simplex method.
2. If the solution is integer, stop; the optimal solution is attained. Otherwise go to step 3.
3. For any updated constraint whose bi value is fractional generate Gomory’s cut. Use dual simplex method to obtain an optimal feasible solution. Then go to Step 2.
k j kj NB j
f x f
s
=
(8)
where
s
0
is a slack variable.Equation 8 is called the (in short f-cut) which should be added to the final tableau of the continuous solution. Since if xj=0;jNB we have s=fk , which is infeasible and thus solved by dual simplex method to restore the feasibility.
3.2.3. Gomory’s All-Integer Cut ([16])
In 1960 Gomory developed a method for solving pure IPP which in the form of steps can be explained as - cut
1. All the coefficients in the original problem are turned integral.
2. Add an artificial constraint such that the sum of the non-basic variables is less than or equal to some arbitrary large number. This makes the problem dual feasible.
3. The new cut generated is pivoted to -1. This will ensure that the tableau will remain integral.
] [ = ] [
k
k j kj
NB j
b s
x
(9)
where >1,sk0 is the slack variable. Equation 9 is required Gomory’s All Integer cut which is also known as
cut.
As xj are non-basic their present value is zero and we have the new basic variable k =[k]= b
s some
negative integer.
Thus, after adding the cut the solution will become infeasible but optimum and thus column dual simplex method is applied to restore feasibility.
3.2.4. Gomory’s Mixed Integer Cut ([18])
Later, Ralph Gomory in his mixed integer cut considered a primal dual starting solution. It is similar to the fractional algorithm but, in this case, not all of the variables are required to be integral.The mixed cut or m-cut is given as below
k j kj J j j k kj k
J j
f w w
f f
) 1 (
(10)
4. NAZ cut ([4])
Bari and Shoib[4] developed a type of cut, NAZ cut which is added to the problem after finding the solution to the LPP. This cut is derived by finding the minimum perpendicular distance from the integer points, which are inside the feasible region, to the objective surface passing through the non-integer solution. The cut is the hyperplane passing through this point and parallel to the objective function surface. The cut has been designed in such a way that the total number of integer solutions in the resulting feasible region is substantially reduced. After adding NAZ cut the problem is then solved by any enumeration technique.
Otherwise let the kth component of x* be non-integer with value xk =ak*. The nearest integer values to k
x are
. , 1,2, = , = 1 ] [ = ]
[
= a* and x a* a* for k n
x k k
k j k
k
j where [t] is the largest integer less than or equal to t and t is the smallest integer greater than or equal to t.
With such bifurcations we can find all the 2n integer points in the surrounding of the non integer solution x*
. Denote the set of indices of these 2n points by T.
Let the objective value at x*be z*. Thus the objective function plane at x* will be
* *
=z
cx (11)
Now the perpendicular distance di from the surrounding points, to the objective plane by using the formula given by Dantzig (1963), is
T i c cx z d
j n
j i
i
, =
2
1 =
0 *
(12)
where xi0 is an integer point around x*. Now we search for the point x0i which has a minimum distance from the objective function hyperplane in the feasible region.
Obviously, the negative distance and the distance from the infeasible points should be omitted and minimum positive distance from the points, which are feasible are chosen.
Let
S
be the set of indices iT for which xi0 are feasible. Let 0={ 0| =min i}S i k
k d d
x x
A plane passing through this integer point and parallel to the objective hyperplane will be cx0=z0 Clearly z0<z*.
We will introduce the NAZ cut
0 0
z
cx (13)
Now z0 acts as a lower bound for the integer solution to the LPP. It has been further proved that this cut will not eliminate the optimal integer solution from the original IPP.
The following theorem will show that this constraint when added to the original problem will not eliminate the integer optimal solution to the IPP.
Theorem: The added constraint will not eliminate the optimal integer solution to the original IPP.
Proof: Let the solution to the original LPP be attained as x*. The cut is derived passing through the integer point
0
x , which is at a minimum positive distance d1 from the objective function level surface passing through the non-integer point x*.
Suppose that x0 is not an integer solution. Then there exists another feasible integer point x** such that .
> 0
* *
z
z (14)
where z**=cx** and z0=cx0 are the values of the objective function at x** and x0 respectively. Let x** be at a distance d2 from
* *
=z cx .
Now we want to show that if z**>z0 then d2<d1. (i.e. x** will be above the cut cx0>z0).
By assumption
. > 0
*
* cx
2 *
1 =
* *
j n
j c cx z
i.e. d2<d1.
Hence the added cut will not exclude the integer optimal solution to the original problem.
This shows that if there is any other integer optimal solution then this will be above the cut not below as depicted in the figure.
The procedure for solving the problem contains following steps: 1. Solve the LPP using simplex or dual simplex method.
2. If this solution is integer, stop. Otherwise, round off the non-integer solution to the nearest integers. 3. Find the minimum perpendicular distance from the integer point, which is inside the feasible region on the objective curve passing through the non-integer solution. Derive NAZ cut passing through this point and parallel to the objective function curve.
4. Use branch and bound or cutting plane method to find the integer optimum.
Bari and Alam[3] further extended this approach to find the integer optimum after adding NAZ cut. They introduced the A-T cut and showed that the integer optimal solution within the reduced region obtained after adding NAZ cut will always lie on this cut. Let x0 be defined as:
0 0 0 0 2 0 1 0
); , , , (
= k S
x nk
k
k
5. A-T Cut ([3])
Now to find the integer optimum solution, A-T cut at x0 is defined as
0 0 0 1 =
= kj
n
S k j n
j
x
The procedure for solving the problem using A-T cut contains following steps:
1. Solve the LP relaxation using simplex method or dual simplex method. If this is integer, stop. Otherwise, go to step (2).
2. We add the NAZ cut which passes through x0 to reduce the feasible region. For finding the integer optimum we add A-T cut at x0 and go to step (3).
3. Prepare the set of integer points Sl at xl1, using the proposed operations. Set, l-1 and check the following conditions to find the next integer point,
(i) It lies on the A-T cut and its component are non-negative.
(ii) The value of the objective function at this point lies between the value objective function at xl1 (integer feasible point) and the value objective function at x* (non-integer solution).
(iii) It satisfies the constraints. And go to step (4).
4. If xl is not satisfying any one of the above conditions then it is outside the reduced feasible region and we
terminate the procedure. It means the previous integer feasible point (xl1) will be integer optimum. Otherwise, set, l=2, and go to step(3).
6. Conclusion
As we know that integer programming problem is one of the most important forms of mathematical programming problems because of its use in wide variety of practical problems and just rounding off the continuous optimal solution is not effective. So there was a need of a method which can provide a best feasible Integer solution. For this many authors have given various methods like branch and bound methods to solve integer programming problems and cutting planes techniques.
In branch and bound method branching is done in which constraints are added such that the total solution space is partitioned and the smaller sub problems are solved for integer solution. The optimal objective value of each sub problem provides a bound (upper for maximization problem) on the objective values of the various integer solutions obtained and we select the best (maximum for maximization problems) out of all Integer Feasible Solutions. The purpose of adding branches is to reduce the complexity of the problems and make it easy to solve. In cutting planes techniques, the purpose is to cut off the infeasible part of the solution space such that no feasible part of the solution space is cut-off.
References
[1] Balas, E. (1968). A note on the Branch and Bound Principle. ICC Bulletin, 16, pp. 442-445.
[2] Balas, E., Ceria, S., Cornuejols, G. and Natraj, N. (1996): Gomory Cut Revisited. Operations Research Letters, 19, pp. 1-9.
[3] Bari, A. and Alam, T. (2005): Search for Integer Optimum after Adding NAZ cut. Pure and Applied Mathematika Sciences, LXII(1-2),
pp. 77-83.
[4] Bari, A., and Shoeb, A. (2003): NAZ Cut for Integer Programming. Pure and Applied Mathematika Sciences, LVII(1-2), pp. 87-94.
[5] Bazaraa, M. S. and Shetty, C. M. (2004): Non-Linear Programming Theory and Algorithm.
[6] Ben-Israel, A. and Charnes, A. (1962): On some Problems in Diophantine Programming. Cah. Cent Etud. Rech. Operationelle, 4, pp.
215-280.
[7] Bender’s, J.F. (1962): Partitioning Procedures for solving Mixed-Variables Programming Problems. Numerische Mathematik, 4, pp.
238-252.
[8] Bertier, P. and Roy, B. (1965): Une Procedure De Resolution Pour Une Classe De Problemes Pouvant Avoir Un Charactere
Combinatoire. ICC Bull., pp. 419-428. (Trans. By W. S. Jewell, ORC Rep. Univ. of California, Berkeley).
[9] Bowman, E. H. and Nemhauser, G. L. Production scheduling problem. Naval Research Logistics, 17(3) pp.309-313, 1970.
[10] Cadoux, F. (2008): Computing Deep Facet-Defining Disjunctive Cuts for Mixed-integer Programming. Mathematical Programming,
Series A, Full Length Paper.
[11] Charnes, A. and Cooper, W.W. (1962): Programming with Linear Fractional Functionals. Naval Research Logistic, 9, pp. 181-186.
[12] Dakin, R.J. (1965): A Tree-Search Algorithm for Fixed Integer Programming Problem. Comput. J. 8, pp. 250-255.
[13] Dantzig, G.B., Fulkerson, D.R. and Johnson, S.M. (1954): Solution of a Large Scale Traveling Salesman Problem. Operations Research,
2, pp. 393-410.
[14] Frangioni, A. and Gentile, C. (2006): Perspective Cuts for a Class of Convex 0-1 Mixed Integer Programs. Mathematical Programming,
106(2), pp. 225-236.
[15] Gomory, R.E. (1958): Outline of an Algorithm for Integer Solutions to Linear Programs. Bull. Amer. Math. Soc., 64, pp. 275-278.
[16] Gomory, R.E. (1960a): All-Integer Integer Programming Algorithm. RC-189. IBM, Yorktown Heights, New York; also in Industrial
Scheduling (J.F. Muth and G.L. Thompson, eds.), pp. 193-206. Printice Hall, Englewood Cliffs, New Jersey, 1963.
[17] Gomory, R.E. (1960b): An Algorithm for Mixed-Integer Problem. RM-2597. RAND Corp. Santa Monica, California.
[18] Granot, D. and Granot, F. (1977): On Integer and Mixed-Integer Fractional Programming Problems. Annals of Discrete Mathematics, 1,
pp. 221-231.
[19] Guignard, M. and Spielberg (1981): Logical Reduction Methods in Zero-One Programming. Operations Research, 29, pp. 49-74.
[20] Harris, P.M.J. (1964): An Algorithm for Solving Mixed-Integer Linear Programs. Operations Research Quarterly, 15, pp. 117-132.
[21] Hoffman, K. and Padberg, M. (1991): Improving LP-Representations of Zero-One Linear Programs for Branch and cut. ORSA Journal of
Computing, 3, pp. 121-134.
[22] Ignizio, J. P., Cavalier, T. M. (1994): Linear Programming, Prentice Hall.
[23] Kesavan, P., Russel J., Edward, P., Gatke, and Borton, P.I. (2004): Outer Approximization Algorithm for Seperable Nonconvex
Mixed-Integer Nonlinear Programs. Mathematical Programming 100(3), pp. 517-535.
[24] Land, A.H. and Doig, A.G. (1960): An Automatic Method for Solving Discrete Programming Problem. Econometrica, 28, pp. 497-520.
[25] Mc Lay, L.A. and Jacobson, S.H. (2007): Integer Knapsack Problems with Set-Up Weights. Computational Optimization and
Applications, 37(1), pp. 35-47.
[26] Lenstra, H. (1983): Integer Programming with a Fixed Number of Variables. Mathematics of O.R., 8, pp. 538.
[27] Markowitz, H.M. and Manne, A.S. (1957): On the Solution of Discrete Programming Problems. Econometrica, 25, pp. 84-110.
[28] Mitten, L.G. (1970): Branch and Bound Methods: General Formulations and Properties. Operations Research, 18, pp. 24-34.
[29] Patel and Chineck (2007): Active constraint variable ordering for faster feasibility of mixed-integer linear programs. Mathematical
Programming, 110(3): pp. 445-474.
[30] Salkin, H. M. (1975): Integer Programming, Addison Wesley Publishing Company.
[31] Savelsberg, H. (1994): Preprocessing and Probing Techniques for Mixed-Integer Programming Problems. ORSA Journal of Computing,
6, pp. 445.
[32] Sierskma, G. (1996): Linear and Integer Programming Theory and Practice, Marcel Dekkar (USA).
[33] Trotter, L.E. and Shetty, C.M. (1974): An Algorithm for the Bounded Variable Pure Integer Programming Problem. Journal of ACM,
21(3), pp. 505-513.
[34] Vyve, V. and Wolsey, L.A. (2006): Approximate extended formulations. Mathematical Programming, 105(2-3), pp. 501-522.
[35] Weismantel, R. (1997): On the 0/1 Knapsack Polytope. Mathematical Programming, 77, pp. 49-68.
