A numerically adaptive implementation of the simplex method

(1)

Introduction Excessively instable problems Results

A numerically adaptive implementation of the

simplex method

József Smidla, Péter Tar, István Maros

Department of Computer Science and Systems Technology University of Pannonia

17th of December 2014.

(2)

Introduction Excessively instable problems Results

Outline

1 Introduction

Pannon Optimizer Numerical errors

2 Excessively instable problems

Hilbert matrix Condition number

Large condition number aware logic

3 Results

Tests

Solving the Hilbert matrix

(3)

Introduction

Excessively instable problems Results

Pannon Optimizer

Optimizer library:

Linear algebraic kernel Auxiliary classes

Primal and dual simplex implementation Alternative sub-algorithms

Our simplex solver can solve the problems of the NETLIB Next steps:

Improve the efficiency

Develop a parallel simplex solver Implement more advanced algorithms Develop and test new solutions Solve excessively instable problems

(4)

Introduction

Linear algebraic kernel

Notations:

Linear programming problem

min cTx Ax_{= b} x_j _{≥ 0, j = 1 . . . n}

Linear algebraic kernel

Provides linear algebraic algorithms and data structures: Vector operations (e.g. dot product)

FTRAN: α = B−1_a

BTRAN: πT _{= h}T_B−1

(5)

Introduction

Numerical errors

Floating point numbers: (−1)s_1.m

1m2. . .mn2e

s _{∈ {0, 1}: sign}

mi: ith bit of the mantissa

e: exponent Errors

Rounding error: |A| » |B|, and B 6= 0 → A = A + B Cancellation: Given A and B 6= 0, A ≈ -B

C = A + B

Expectation: C = 0 Error: C = ±ε

These errors can create a lot of fake nonzeros, lead to wrong results and slow down the solution process.

(6)

Introduction

Relative errors

x and y are floating point numbers, x, y > 0 δx : relative error of x

Relative error of the summation:

δx +y = max{δx, δy}

Relative error of the subtraction: δx−y = x x_{− y}δx+ y x_{− y}δy, where x > y

If x − y < 1 the relative error is amplified! Relative errors can accumulate.

(7)

Introduction

Excessively instable problems

Results

Hilbert matrix

Hilbert matrix: Hn,n, where hi,j = _{i +j−1}1 , ∀i, j = 1, . . . , n

Example: H_4,4 ₌       1 1₂ 1₃ 1₄ 1 2 13 14 15 1 3 14 15 16 1 4 15 16 17      

We can construct the following LP problem: min 0 H_n,nx_{= b} xj ≥ 0, j = 1 . . . n, and bi = n X j =1 1 i_{+ j − 1}

(8)

Introduction

Results

Open source solvers and the Hilbert matrix

It is clear that if and only if xj = 1, j = 1 . . . n, the solution is

optimal

We have tested CLP and GLPK

Size GLPK Exact GLPK CLP 3× 3 xj = 1 ± 3.997 ∗ 10−15 xj = 1 xj = 1 4× 4 xj = 1 ± 8.271 ∗ 10−13 xj = 1 xj = 1 5× 5 xj = 1 ± 1.75 ∗ 10−11 xj = 1 xj = 1 6× 6 xj = 1 ± 2.661 ∗ 10−10 xj = 1 INFEASIBLE 7× 7 xj = 1 ± 1.157 xj = 1 INFEASIBLE 8× 8 xj = 1 ± 1.6001 xj = 1 ± 0.201 INFEASIBLE 20× 20 xj = 1 ± 6.298 xj = 1 ± 4.124 INFEASIBLE 100× 100 0≤ xj ≤ 24.009 xj = 1 ± 21.682 INFEASIBLE

We have used CLP ang GLPK as libraries, the models were generated and solved by C++ programs.

(9)

Introduction

Results

Condition number

Measures, how much the output changes if the input changes κ_{(A) = kA}†_{k × kAk, where A}† = V

_S₋₁ 0

0 0

UT_,

and the singular value decomposition of A = U

S ₀ 0 0

VT

If κ(A) is large, computing A† is difficult

The condition number of the n*n Hilbert matrix is very large, it grows as O (1 + √ 2)4n √_n ! κ(H10,10) ≈ 3.536 ∗ 1013 κ(H100,100) ≈ 2.42 ∗ 10148

(10)

Introduction

Results

Exact solver

Idea: use an exact arithmetic solver

For example: open source tools, like mpf_class or mpq_class Problem: too slow → use adaptive methods!

One hour on this planet is 7 years

on Earth.

Great, we’ll wait here until the solver finishes.

(11)

Introduction

Results

Compute the condition number

Computing the condition number is difficult

Assume, there exists an efficient and reliable algorithm for computing the condition number

The condition number of the following matrix is large, but the problem is stable: min 2n X j =n+1 xj [I|H] = b xj ≥ 0, j = 1 . . . 2n, and bi = 1

If the starting basis is the unit matrix, columns of H will never enter to the basis → if the condition number of the matrix is large, it does not necessary causes troubles

(12)

Introduction

Results

Control logic

Requirements:

Recognize the instable bases Adjust the precision if necessary

If the problem is stable, the overhead of monitoring the problem must be close to zero

Return to lower precision arithmetic if the basis becomes stable Solution:

Primary large condition number detector: heuristic, quick algorithm, indicates that the current basis is instable Secondary large condition number detector: slow, but exact sensitivity analysis

The primary detector minimizes the frequency of running the secondary detector → minimizes the overhead

(13)

Introduction

Results

Primary large condition number detector

We can not compute the condition number directly However, we can detect the effect of the large condition number

We propose:

The input of the classic FTRAN is vector a: B−1a_{= α} Create the perturbed ¯a_{copy of a}

Use a modified FTRAN, which computes B−1_a_{= α and}

B−1_¯a_{= ¯}α

The modified FTRAN perturbs every sum during the computation of ¯α

If r = max{kαk,k ¯αk}

min{kαk,k ¯αk} is greater than a threshold, it means that

the condition number is too large → primary alarm

(14)

Introduction

Results

Large condition number aware logic

The primary detector is executed if

an error occurs, for example: fallback to phase-1

If a primary alarm occurs, the algorithm performs primary detector in the following iterations

If primary alarms occur in every next iteration and r does not decrease→ secondary alarm

the algorithm terminates

If a primary alarm occurs, the algorithm performs a sensitivity analysis

If the sensitivity analysis finds that the result is extremely instable→ secondary alarm

If a secondary alarm occurs, the software restarts from the last basis with modified parameters (enabled scaling, switching to LU decomposition, etc.)

In the last resort: The software restarts from the last basis with enhanced precision arithmetic

(15)

Introduction Excessively instable problems

Results Tests

Primary large condition number detector

Tests

The tests were performed on the following environment: CPU: Intel(R) Core(TM) i5-3210M CPU @ 2.50GHz Memory: 8 GiB

Operating system: Debian 7, 64 bit Tests with the following Open MP types:

multiple precision floating point arithmetic: mpf_class 128 bit

mpf_class 256 bit mpf_class 512 bit

rational number arithmetic: mpq_class

The output of the primary detector has been tested, too

(16)

Results Tests

Solving the Hilbert matrix: mpf_class

(17)

Results Tests

Solving large scale Hilbert matrix

time unit: seconds

mpq_class: rational number arithmetic is the most precise but the slowest

Size mpf_class 128 mpf_class 256 mpf_class 512 mpq_class

80 0.166 0.172 0.193 6.67

90 0.242 0.2428 0.273 9.92

100 0.324 0.3449 0.375 14.2

500 38.83 40.5 45.53 4038.9

1000 315 322.17 368.19 ?

(18)

Results Tests

Solving the Hilbert matrix: solutions

Size IEEE 754 mpf_class 128 bit

3 xj = 1 ± 1.044 ∗ 10−14 xj = 1 ± 2.449 ∗ 10−40 4 xj = 1 ± 1.517 ∗ 10−13 xj = 1 ± 2.457 ∗ 10−40 5 xj = 1 ± 6.395 ∗ 10−13 xj = 1 ± 2.379 ∗ 10−40 6 xj = 1 ± 4.369 ∗ 10−10 xj = 1 ± 1.635 ∗ 10−39 7 INFEASIBLE xj = 1 ± 2.478 ∗ 10−40 8 INFEASIBLE xj = 1 ± 2.372 ∗ 10−40 20 INFEASIBLE xj = 1 ± 6.016 ∗ 10−25 100 INFEASIBLE xj = 1 ± 8.983 ∗ 10−24 500 INFEASIBLE xj = 1 ± 3.467 ∗ 10−22 1000 INFEASIBLE xj = 1 ± 7.340 ∗ 10−22

(19)

Results Tests

Primary large condition number detector

Output of the detector: r = max{kαk,k ¯αk} min{kαk,k ¯αk}

δ _{= r − 1}

Problem Value of δ after the last iteration

25FV47.MPS 3.66059e-08 STOCFOR3.MPS 3.07735e-08 PILOT.MPS 9.22276e-06 MAROS-R7.MPS 1.39086e-10 Hilbert 7*7 0.0167145 Hilbert 8*8 0.524724 Hilbert 20*20 2.05845 Hilbert 26*26 5.45188 Hilbert 100*100 0.323612

(20)

Results Tests

Conclusions, next steps

The primary and secondary detectors can minimize the overhead of the adaptivity

Multiple precision arithmetic can be used for the excessively instable problems

We have to integrate the enhanced precision arithmetic to the Pannon Optimizer

We have to integrate the large condition number recognizer algorithm

The large condition number recognizer can be accelerated with low-level optimization (SIMD architecture)

Our goal: Implement a solver which runs fast on the stable problems, but recognizes the excessively instable problems → Switches to more precise arithmetic, and solves these problems too

(21)

Results Tests

Thank you for your attention!

The research and publication has been supported by the European Union and Hungary and co-financed by the European Social Fund through the project

TAMOP-4.2.2.C-11/1/KONV-2012-0004 - National Research Center for Development and Market Introduction of Advanced Information and Communication Technologies.