Experiments - Decision procedures for linear arithmetic

We have found instances of absolutely unbounded polyhedra in some classes of the SMT-LIB benchmarks [10]. These instances are 229 of the 233 dillig benchmarks designed by Dillig et al. [53], 503 of the 591 CAV-2009 benchmarks also by Dillig et al. [53], 229 of the 233 slacks benchmarks which are the dillig benchmarks extended with slack variables [88], and 19 of the 37 prime-conebenchmarks, that is, “a group of crafted benchmarks encoding a tight n-dimensional cone around the point whose coordinates are the first n prime numbers” [88]. The remaining problems (4 from dillig, 88 from CAV-2009, 4 from slacks, and 18 from prime-cone) are not absolutely unbounded because they are either tightly bounded or integer unsatisfiable. For our experiments, we look only at the instances of those benchmark classes that are actually absolutely unbounded.

Using these benchmark instances, we have confirmed our theoretical as- sumptions (Lemma 4.4.1) in practice. We integrated the unit cube test into our own branch-and-bound solver SPASS-IQ (v0.3)2 and ran it on the absolutely unbounded instances; once with the unit cube test turned on (SPASS-IQ+uc) and once with the test turned off (SPASS-IQ). For every problem, SPASS-IQ+uc applies the unit cube test exactly once. This application happens before we start the branch-and-bound approach. To evaluate the efficiency of the unit cube test, we compared SPASS-IQ to several other solvers for systems of linear inequalities.

Benchmark Name CAV-2009 DILLIG PRIME-CONE SLACKS ROTATE

#Instances 503 229 19 229 229

Solvers: solved time solved time solved time solved time solved time SPASS-IQ+uc 503 7.8 229 3.6 19 0.0 229 6.1 229 4.0 SPASS-IQ 502 156 229 39 19 0.1 220 64 229 8.4 Ctrl-Ergo 503 9.7 229 4.2 19 0.1 229 20.2 228 185884 CVC4 426 19367 195 10646 19 1.4 138 1268 191 6062 MathSAT5+uc 503 39 229 17 19 0.2 229 38 229 19 MathSAT5 502 8607 228 3530 19 3.3 192 18055 229 1464 SMTInterpol 493 11348 225 3666 19 14 206 15975 178 4120 Yices 477 44236 211 11586 19 0 154 24525 198 52679 Z3+uc 503 281 229 97 19 0.1 229 100 229 97 Z3 472 3049 214 1639 19 0.1 160 368 214 1634

Figure 4.8: Experimental Results: SMT solvers

Comparison with State-Of-The-Art SMT Solvers

First, we compared SPASS-IQ with state-of-the-art SMT solvers for linear integer arithmetic: CVC4 (v1.6) [9], MathSAT5 (v5.5.2) [42], SMTInterpol (v2.5-19) [41], Yices (v2.6.0) [57], and Z3 (v4.8.1) [50]. All these solvers employ a branch-and-bound approach with an underlying dual simplex solver [58]. The only exception are MathSAT5 and Z3, which, subsequent to our first publication on the unit cube test [35], now also perform the unit cube test in advance. That is why we also test MathSAT5 and Z3 once with the unit cube test turned on (MathSAT5+uc and Z3+uc) and once with the test turned off (MathSAT5 and Z3).

The solvers had to solve each problem in under 40 minutes. For the experiments, we used a Debian Linux cluster and allotted to each problem and solver combination 2 cores of an Intel Xeon E5620 (2.4 GHz) processor, 4 GB RAM, and 40 minutes. Figure 4.8 lists the results of the different solvers (column one) on the different benchmark classes (row one). Row two lists the number of benchmark instances we considered for our experiments. For each combination of benchmark class and solver, we have listed the number of instances the solver could solve in the given time as well as the total time (in seconds) of the instances solved (columns labelled with “solved” and “time”, respectively).

Our solver that employs the unit cube test solves all instances with the application of the unit cube test and is 10 times faster than our solver without the test. The SMT theory solvers in their standard setting were not able to solve all instances within the allotted time. Moreover, our unit cube test was over 100 times faster than any state-of-the-art SMT solver without the unit cube test. The results for MathSAT5 and Z3 further support the superiority of the test.

Comparison with Ctrl-Ergo

Second, we compared our unit cube test with the Ctrl-Ergo solver, which in- cludes a subroutine that is essentially the dual to our largest cube test [26]. As expected, both approaches are comparable for absolutely unbounded polyhedra. In order to also compare the two approaches on benchmarks that are not absolutely unbounded, we created the rotate benchmarks by adding the same four inequalities to all absolutely unbounded instances of the dillig benchmarks. These four inequalities essentially describe a square bounding the variables x0 and x1 in an interval [−u, u]. For a large enough choice of u (e.g., u = 210), the square is so large that the benchmarks are still integer satisfiable and not absolutely trivial for branch-and-bound solvers. To add a challenge, we rotated the square by a small factor 1/r, which resulted in the following four inequalities:

−b · r · r + r ≤ b · r · x0− x1 ≤ b · r · r − r , and −b · r · r + r ≤ x0+ b· r · x1 ≤ b · r · r − r .

These changes have nearly no influence on SPASS-IQ, and two SMT solvers even benefit from the proposed changes. For Ctrl-Ergo the rotate benchmarks are very hard because its subroutine detects only absolutely unbounded polyhedra. If the polyhedron is not absolutely unbounded, then Ctrl-Ergo starts its search from the boundaries of the polyhedron instead of looking at the polyhedron’s interior. We can even control the number of iterations (r2) Ctrl-Ergo spends on the parts of the boundary without any integer solutions if we choose r accordingly (e.g., r = 210_{). In contrast, we} use our cube tests to also extract interior points for rounding. This difference makes our tests much more stable under small changes to the polyhedron.

Most problems in the linear integer arithmetic SMT-LIB benchmarks with finite lattice width (i.e., that are not absolutely unbounded) can be solved without using any actual integer arithmetic techniques. A standard simplex solver for the rationals typically finds a rational solution for such a problem that is also an integer solution. Applying the unit cube test on these trivial problem classes is a waste of time. In the worst case, it doubles the eventual solution time. For these examples it is beneficial to first compute a general rational solution and to check it for integer satisfiability before applying the unit cube test. This has the additional benefit that rational unsatisfiable problems are filtered out before applying the unit cube test. The unit cube test is also guaranteed to fail on problems containing boolean variables, i.e., variables that are either 0 or 1, unless they are absolutely trivial and describe a unit cube themselves. Whenever the problem contains a boolean variable, it is beneficial to skip the unit cube test. This is also the reason why we provide no experimental results for the theory of linear mixed arithmetic, i.e., the few mixed benchmarks available in the SMT-LIB all contain boolean variables.

Benchmark Name CAV-2009 DILLIG PRIME-CONE SLACKS ROTATE

#Instances 503 229 19 229 229

Solvers: solved time solved time solved time solved time solved time SPASS-IQ+uc 503 7.8 229 3.6 19 0.0 229 6.1 229 4.0

SPASS-IQ 502 156 229 39 19 0.1 220 64 229 8.4

GLPK 503 24 229 13 19 0.0 121 4.3 229 9.8

Gurobi 503 3.7 229 1.7 19 0.1 229 1.6 229 0.4

SCIP 503 42 229 19 19 0.1 224 34 229 16

Figure 4.9: Experimental Results: MILP solvers

Comparison with MILP Solvers

Third, we compared our unit cube test with several solvers for mixed- integer programming (MILP) (see Figure 4.9): the two non-commercial solvers GLPK (v4.65) [107] and SCIP (v6.0.0) [71] as well as the commercial solver Gurobi (v7.52) [78]. For these experiments, we used the same benchmarks—although converted into the MPS (Mathematical Program- ming System) format—and the same experiment parameters as for our experiments with the SMT solvers. In General, mixed-integer programming solvers have an advantage over standard SMT theory solvers because (i) they are not required to be exact and sound, which allows them to use floating- point arithmetic, and (ii) they are not required to be incrementally efficient, which means they can use much more elaborate techniques. Despite these advantages, SPASS-IQ is faster and solves more problems from the absolutely unbounded benchmarks than GLPK and SCIP. The reason is that GLPK and SCIP rely—like the state-of-the-art SMT theory solvers—on a branch-and-bound approach with an underlying simplex solver, which means they also focus their search on the vertices of the polyhedron instead of the polyhedron’s interior. Gurobi, on the other hand, is faster than SPASS- IQ on the absolutely unbounded benchmarks because (i) it uses an interior point method [94] to compute the first rational solution for its branch-and- bound approach and only then switches to its simplex solver and (ii) it uses floating-point arithmetic, which is more efficient than exact arithmetic.

The experiments with Gurobi give the impression that interior point methods are an efficient alternative to our unit cube test. At least for now, this impression is only correct for mixed-integer programming and not for SMT theory solvers. Interior point methods perform worse in the context of SMT theory solvers because the currently competitive interior point methods are not incrementally efficient, which is one of the most important properties for an efficient SMT theory solver [62].

On Other SMT-LIB Benchmarks

4.6 Summary

We have presented the linear cube transformation (Corollary 4.2.2), which allows us to efficiently determine whether a polyhedron contains a cube of a given edge length. Based on this transformation we have created two tests for linear integer arithmetic: the largest cube test and the unit cube test. Our tests can be integrated into SMT theory solvers without sacrificing the advantages that SMT solvers gain from the incremental structure of subsequent subproblems. Furthermore, our experiments have shown that these tests increase efficiency on certain polyhedra such that previously hard sets of constraints become trivial.

Chapter 5

Computing a Complete Basis

for Equalities (Implied by a

System of LRA Constraints)

Equalities are a special instance of linear arithmetic constraints. They are useful in simplifying systems of arithmetic constraints [77], and they are essential for the Nelson-Oppen style combinations of theories [37, 112, 119]. However, they are also an obstacle for our fast cube tests (see Chapter 4). If an inequality system implies an equality, then it has only a surface and no interior; so our cube tests cannot explore an interior and will certainly fail. In order to expand the applicability of our cube tests, we designed methods that find, isolate, and eliminate implied equalities from systems of linear arithmetic constraints.

We can detect the existence of an implied equality by searching for a hypercube in our polyhedron. If the maximal edge length of such a hypercube is zero, then there exists an implied equality. This test can be further simplified. By turning all inequalities into strict ones, the interior of the original polyhedron remains while the surface disappears. If the strict system is unsatisfiable, then the original system has no interior and implies an equality. Moreover, the method generates an implied equality as a proof based on an explanation of unsatisfiability for the strict system.

We are also able to extend the above method into an algorithm that computes an equality basis, i.e., a finite representation of all equalities implied by a satisfiable system of inequalities. For this purpose, the algorithm repeatedly applies the above method to find, collect, and eliminate equalities from our system of constraints. When the system contains no more equalities, then the collected equalities represent an equality basis, i.e., any implied equality can be obtained by a linear combination of the equalities in the basis. The equality basis has many applications. If transformed into a substitution, it eliminates all equalities implied by our system of constraints,

which results in a system of constraints with an interior and, therefore, im- proves the applicability of our cube tests. The equality basis also allows us to test whether a system of linear arithmetic constraints implies a given equality. We even extend this test into an efficient method that computes all pairs of equivalent variables inside a system of constraints. These pairs are necessary for the Nelson-Oppen style combination of theories.

This chapter is organized as follows: In Section 5.2, we show how to investigate equalities with the linear cube transformation. We do so by first introducing an efficient method for testing whether a system of linear arithmetic constraints implies a given equality (Section 5.2.1); then, we extend the method so that it computes an equality basis for our system of constraints (Section 5.2.2). In Section 5.3, we describe an implementation of our methods as an extension of Dutertre and de Moura’s version of the simplex algorithm, which is integrated in many SMT solvers (see Chap- ter 2.7 and [58]). The implementation generates justifications and preserves incrementality. The efficient computation of an equality basis can then be used in identifying equivalent variables for the Nelson-Oppen combination of theories (Section 5.4). This is also the first application of the equality basis that we discuss in depth. The second application we present uses the equality basis for the computation of a bounded basis (Definition 2.8.5) and for the detection of bounded and unbounded directions (Section 5.5). The final application we present uses the equality basis for quantifier elimination (Section 5.6). Section 5.7 concludes the chapter with a summary of the presented results.

5.1 Related Work and Preliminaries

This chapter is based on two publications with Christoph Weidenbach as co-author [34, 36]. Only Section 5.5 has never been published before this thesis.

This chapter focuses like the previous chapter on the geometric inter- pretation of systems of inequalities. This is the reason why we also use in this chapter polyhedron as an alternative name for systems of inequalities Ax_{≤ b. The other geometric objects we are considering are cubes. Their} definition can be found in Chapter 2.9.

This chapter also focuses on the theory of linear rational arithmetic and not on the more general theory of linear mixed arithmetic. Therefore, we abbreviate in this chapter rational satisfiability/equivalence/entailment with satisfiability/equivalence/entailment. Nonetheless, some of the applications we present verge into the theory of linear mixed arithmetic. To avoid confu- sion, we always specify the type of solution/satisfiability if it is not rational.

The constraints in this chapter are non-strict inequalities and they are either formatted according to the vector representation, i.e., aT

i x ≤ bi (see also Chapter 2.2.1), or the standard representation, ai1x1+ . . . + ainxn≤ bi (see also Chapter 2.2.1). This chapter also deals with equalities aT

i x = bi besides non-strict inequalities. However, an equality is just our way of highlighting the set of inequalities _{aT

i x≤ bi,−aTi x≤ −bi} as explained in Chapter 2.2.1. Other types of constraints have to be reduced to non-strict inequalities with the techniques presented in Chapter 2.3.

This chapter builds on the basics of linear algebra (Chapter 2.1) and linear arithmetic (Chapter 2.2), on the concept of implied constraints (Chap- ter 2.5), and on the definitions of (un)bounded and (un)guarded problems and variables (Chapter 2.8). Our example implementation (Section 5.3) also builds on the notions and definitions of standard arithmetic decision procedures for SMT solvers as presented in Chapter 2.7.

There also already exist several methods that find, isolate, and eliminate implied equalities [21, 124, 133, 82]. Hentenryck and Graf [82] define unique normal forms for systems of linear constraints with non-negative variables. To compute a normal form, they first eliminate all implied equalities from the system. To this end, they determine the lower bound for each inequality by solving one linear optimization problem. Similarly, Refalo [124] descri- bes several incremental methods that use optimization to turn a satisfiable system of linear constraints in “revised solved form” into a system without any implied equalities. Rueß and Shankar also use this optimization scheme to determine a basis of implied equalities [126]. Additionally, they present a necessary but not sufficient condition for an inequality to be part of an equality explanation. During preprocessing, all inequalities not fulfilling this condition are eliminated, thus, reducing the number of optimization problems their method has to compute. However, this preprocessing step might be in itself expensive because it relies on a non-trivial fixed-point scheme. The method presented by Telgen [133] does not require optimization. He presents criteria to detect implied equalities based on the tableau used in the simplex algorithm, but he was not able to formulate an algorithm that efficiently computes these criteria. In the worst case, he has to pivot the simplex tableau until he has computed all possible tableaux for the given system of constraints. Another method that detects implied equalities was presented by Bjørner [21]. He uses Fourier-Motzkin variable elimination to compute linear combinations that result in implied equalities.

Our methods that detect implied equalities do not require optimization, which is advantageous because SMT solvers are usually not fine-tuned for optimization. Moreover, we defined our methods for a rather general formu- lation of linear constraints, which allows us to convert our results into other representations, e.g., the tableau-and-bound representation used in Duter- tre and de Moura’s version of the simplex algorithm (see Section 5.3), while

preserving efficiency. Finally, our method efficiently searches for implied equalities. We neither have to check each inequality independently nor do we have to blindly pivot the simplex tableau. This also makes potentially expensive preprocessing techniques obsolete.

In document Decision procedures for linear arithmetic (Page 155-164)