The Branch-and-Cut Algorithm

The branch-and-bound algorithm is the standard method for solving integer programs. Many of today’s ILP solvers employ an extended variant—branch-and-cut—which is outlined in the following.

The algorithm starts with solving the LP-relaxation. If the result is fractional-valued, branch-and-cut combines two different approaches to obtaining integral solutions: the first tightens the polytope by deriving and adding further constraints (cuts), while the second successively parti-tions the polytope into smaller subpolytopes and tackles them separately (branching).

1 2 3 4 1

2 3 x2

x1 1 2 3 4

1 2 3 x2

x1

Figure 4.4: Illustration of Example 1.

The left-hand side shows the feasible region of the LP with its optimal solution (at the cross).

The right-hand side shows the same problem as an ILP; only after adding the Gomory-Chvátal cutx1+ x2 ≥ 3 the LP-relaxation has an integer feasible optimal solution (2, 1).

A cutting plane (cut) is an inequality that is valid for PI but not necessarily for PR, i.e., that cuts away a subset of non-integral solutions. Usually, this subset includes the current (non-integral) optimal solution so that reoptimizing is necessary afterwards to obtain a feasible solution again. As each added inequality increases the size of the constraint matrix, cuts should be applied selectively. The process is usually guided by heuristics that estimate the effectiveness of possible cuts.

The ILP solver CPLEX automatically adds cuts from up to nine different classes, depending on the structure of the ILP [ILO03b]. The following two were most often applied to the ILPs generated in this work:

• Clique cuts: A group of binary variables x₁, . . . , xk forms a clique if at most one of these variables can be non-zero in any integer feasible solution. Then the inequality_k

i=1xi ≤ 1 is valid and under certain conditions also facet-inducing (see Sec. 4.3). This relationships can be tracked by ILP solvers via a clique table and dynamically exploited to derive cuts.

• Gomory fractional cuts: The idea behind these classical cuts [Gom58] can be demonstrated on the basis of the similar Gomory-Chvátal cutting planes: For any inequalitya^Tx ≤ δ that is valid forP^R and wherea ∈ Zⁿ, the Gomory-Chvátal cutting planea^Tx ≤ δ is valid for all integral points in PR, and thus valid for PI [Eis00, Sch86]. Figure 4.4 illustrates such a cut obtained from the valid inequalityx1+ x2 ≥ 2.5 of Example 1.

Gomory cuts are considered the most general as for any fractional-valued optimal solution of the relaxation a violated Gomory cut can be found; the successive addition of these cuts always leads to an optimal integral solution [NW88]. However, this result is, taken alone, of little value

in practice since the convergence occurs much too slowly. Nevertheless, Gomory cuts have turned out to be effective if their use is integrated with the branch-and-cut process and if they are not added one at a time, but in groups [BFG⁺00].

If the addition of cuts still does not produce an integer feasible optimal solution, a decomposi-tion into smaller subproblems takes place which are recursively solved using the same algorithm.

The solution of the whole problem is then obtained by combining the solutions of the subpoly-topes as described by the following proposition:

Proposition 4.2.6 (Divide and Conquer) Let a minimization problem be given with feasible re-gionP and objective function c : P −→ R, and let P1, . . . , Pk be a partition ofP , i.e., Pi ⊂ P and
k

i=1Pi = P . If we define zQ= min {c(x) |x ∈ Q} for each subset Q ⊆ P , then

zP = min {zP₁, . . . , zP_k} _✷ Since the decomposition and solution of the subpolytopes occurs recursively, a decomposi-tion tree is formed in which each subproblem is a node. The active subproblems—those which still require further processing—are to be found in the leaves of this branch-and-bound tree. The partitioning usually is performed according to Dakin [Dak65]: The algorithm selects a decision variable with fractional valueδ and branches on this variable by creating two subproblems with additional boundsxⁱ ≤ δ and xi ≥ δ!, respectively. For binary variables, this means that they are fixed at 0 and 1, respectively.

Carried to the extreme, this decomposition could lead to a total enumeration of the integer feasible points, which is not viable. Hence the tree is continuously pruned, i.e., certain submodels are discarded from further processing (fathomed). For this purpose, the algorithm always retains the best integer solution found so far as the incumbent (solution). Its objective function value, denoted byz, is a monotonously decreasing upper bound on the objective function value of an integer feasible optimal solution. Subproblems are fathomed if they cannot contain an integral solution with an objective function value less than this bound. The incumbent is known to be globally optimal if no active subproblem remains, i.e., if for all subpolytopesPi of the leaves of the tree

1. an optimal integral solution ofPiis known, or

2. it is known thatPi does not contain an integral solution at all, or

3. it is known that all integral solutions ofPihave an objective function value greater or equal toz.

The occurrence of the second and the third case can be determined efficiently by solving the LP-relaxation of the subproblem or its dual. Regarding the second case, a subpolytopePi does not contain an integral solution if

• its LP-relaxation is infeasible—then there can also be no integral solution,

• or if the dual of its LP-relaxation is unbounded—then it follows from duality (Theorem 4.1.6) thatPi is empty.

Regarding the third case, we observe that for the objective function value zI of any integer feasible solution inPi it holdszI ≥ z if

• the LP-relaxation has an objective function value zR ≥ z—then it follows with Remark 4.2.3zI ≥ zR≥ z,

• or if the dual of its LP-relaxation has a feasible solution zD ≥ z—then it follows from dualityzI ≥ zR≥ zD ≥ z.

The latter point gives one reason why branch-and-bound implementations usually solve the dual of the LP-relaxation at each node (or, equivalently, apply the dual simplex algorithm directly to the primal): While only an optimal solution of the primal delivers a lower bound, every feasible solution of the dual already does the same. The greatest lower bound obtained this way is often referred to as the “best bound” of the node. This value is monotonously increasing with the depth of the nodes in the tree.

1 2 3 x1 4

1 2 3 x2

1 2 3 x1 4

1 2 3 x2

Figure 4.5: Application of branch-and-bound to Example 1.

Figure 4.5 depicts an example for a simple branch-and-bound procedure. It starts with an optimal solution of the LP-relaxation as shown in the left-hand side of Fig. 4.4 (intentionally no cuts are added). The first partitioning is x2 ≤ 1.5 and x₂ ≥ 1.5!, yielding the integer feasible solution(2, 1) with objective function value 3 in the first of the two subpolytopes (left-hand side of Fig. 4.5). However, the best bound ⁸₃ of the other subpolytope does not allow to exclude the possibility that a better integer feasible solution can still be found here². Thus a

2At least for the described algorithm. A more sophisticated solver could observe that the objective function has only integral coefficients and that consequently the objective function value of every integer feasible solution is integral. This allows to round the best bound up to3, to fathom the subproblem and to abort the branch-and-bound process already here.

further decomposition of this subpolytope occurs (x1 ≤ ₂

3

and x1 ≥ ₂

3

, right-hand side), yielding the best bound3 in both resulting subproblems. Hence (2, 1) is an optimal solution of the ILP. The corresponding branch-and-bound tree is depicted in Fig. 4.6.

Optimal Solution: (1,1.5) Best Bound: 2.5

Optimal Solution: (2,1)

Best Bound: 3 Optimal Solution: (0.6,2) Best Bound: 2.6

Optimal Solution: (0,3)

Best Bound: 3 Optimal Solution: (1,2) Best Bound: 3 x₂³ 2

x₂£ 1

x₁³ 1 x₁£ 0

Root node

(*)

(*)

Figure 4.6: Branch-and-bound tree corresponding to Fig. 4.5.

Nodes marked with (*) have subpolytopes consisting only of a single integer vector.

From the description of the branch-and-bound process it becomes clear why a tight ILP for-mulation is so crucial to efficient solvability: Firstly, then more of its extreme points are integral, or have more integral components. Secondly, the bound from the LP-relaxation gets tighter and allows to prune the branch-and-bound tree earlier. This helps to contain its growth, which can be exponential in the worst case.

Besides the tightness, two major sorts of decisions during the branch-and-bound process strongly influence this growth. The first decision concerns the choice of the next active node for branching. After processing a node, the ILP solver can

• continue to dive deeper into the tree from there by selecting one of the two newly created successors of this node—if available—for decomposition (depth-first search),

• or select one of the other remaining active nodes and proceed there (backtracking).

The first choice prioritizes the early finding of integer feasible solutions, which are more likely to be found deep in the tree than at nodes near the root [NW88, ILO03b]. The second works at moving the best bounds, which is essential to proving that an attained incumbent is optimal, i.e., that no better integer solution can be found in another active subproblem. ILP solvers balance these two sometimes-competing aims heuristically. Descriptions of backtracking strategies can be found in [NW88, ILO03b]; they can be parameterized by the user to emphasize either early feasibility or proving optimality.

During backtracking, a subproblem has to be selected among all active leaf nodes. The most common strategy chooses the node with the lowest best bound, which is usually one near the top of the tree. In contrast to this, an alternative strategy chooses a node that is estimated most likely to contain an optimal solution (best estimate), which generally means diving deeper into the tree from a different leaf.

The second decision concerns the selection of a fractional-valued variable for branching at a node. It is preferable to choose variables whose integrality would also implicate the integrality of other variables. However, robust methods for identifying such variables have not been estab-lished, therefore it is common to let the user specify priorities [NW88]. [ILO03b] recommends that, if integer variables represent different types of decisions, and if a decision depends on an-other, then variables representing the depending decision should have a lower priority than those representing the other.

If no priority order is available or if there is more than one variable with maximal priority, then there exist a variety of heuristics for the choice of a variable; several are described in detail in [NW88, ILO03b]. It is also possible to predetermine the branch direction of a variable, i.e., which of the two subproblems resulting from the division should be explored first.

In modern ILP solvers, the branch-and-cut process is supported by preprocessing and node heuristics. Preprocessing applies several reductions to the ILP in order to simplify constraints, reduce the problem size, and eliminate redundancy. This can include tightening bounds, fixing and substituting variables, and removing redundant constraints [NW88, ILO03b]. In CPLEX, these operations are applied iteratively in several passes to the root node prior to solving it and—

in a restricted way—also to the subproblems within the tree (node presolve). They can greatly facilitate the solution process (see Sec. 7.3.1, [NW88]).

Node heuristics try to find integer feasible solutions at nodes during the branch-and-cut pro-cedure heuristically. These periodically applied routines usually start from an available feasible solution and fix successively fractional-valued variables to integers, subject to the condition that the vector remains feasible and has an objective function value less than the incumbent’s one.

If an integral vector results, a new incumbent has been found and the node heuristics was suc-cessful. Since successively better incumbents are highly important to move the upper bound and prune the tree through this, the computational effort of the node heuristics pays even if it succeeds only in a small fraction of all cases.