Algorithm

First, we describe the outer algorithm. Let ε0 > 0 be a tolerance.

Algorithm 1. An incremental algorithm

1: ( Initialization) Set A1_u = Au, Q1u = ∅, u = 1, . . . , q. Select any starting point

(x, y) such that x ∈ IRn, y ∈ IR1 and set

X1u = x, Y1u = y, ∀ u = 1, . . . , q.

Set

C1 = {1, . . . , q}, I1u = {1}, J11u = {1}, r1u= 1, s11u= 1, u = 1, . . . , q,

the number of hyperplanes for class u: l1u= 1 and iteration counter k = 1.

2: ( Stopping criterion) If |Ck| ≤ 1 then stop. Otherwise go to Step 3.

3: ( Computation of piecewise linear functions) For each u ∈ Ck apply Algo-

rithm 2. This algorithm generates a piecewise linear boundary (Xku∗_{, Y}

ku∗),

a starting point (Xk+1,u_{, Y}

k+1,u) ∈ IR(n+1)lk+1,u for class u, the set Ak+1u con-

taining “undetermined” points and the set Qk

u of easily separated points from

class u.

4: ( Refinement of set Ck) Refine the set Ck as follows:

Ck+1 = {u ∈ Ck: |Ak+1_u | > ε0|Au|}.

Set k = k + 1 and go to Step 2.

We will now present the inner algorithm for separating class Au, u ∈ {1, . . . , q}

from the rest of the data set. At each iteration k of Algorithm 1 we get the subset Ak

u ⊆ Au of the set u ∈ Ck which contains points from this class which are not

easily separated using piecewise linear functions from previous iterations. Let

Qk_u = [

j=1,...,k

Qj_u

be a set of all points removed from the set Au during the first k > 0 iterations.

We denote Dk= [ t=1,...,q  At\ Q k t  , Ak_u = [ t=1,...,q,t6=u  At\ Q k t  .

Algorithm 2 finds a piecewise linear function separating the sets Ak

u and Aku.

Let ε1 > 0, ε2 > 0, ε3 > 0 be given tolerances and σ ≥ 1 be a given number.

Algorithm 2. Computation of piecewise linear functions Input Starting points (Xku_{, Y}

ku) ∈ IR(n+1)lku, the set of indices Iku, Jiku, i ∈

Iku and the number of hyperplanes lku at iteration k of Algorithm 1.

Output A piecewise linear boundary (Xku∗_{, Y}

ku∗) ∈ IR(n+1)lku, the set of in-

dices Ik+1,u, Jik+1,u, i ∈ Ik+1,u, a number of hyperplanes lk+1,u, a starting

point (Xk+1,u_{, Y}

Ak+1

u and a set Qk+1u of easily separated points from class u.

1: ( Finding a piecewise linear function) Solve problem (3.17) over the set D_k

starting from the point (Xku_{, Y}

ku) ∈ IR(n+1)lku. Let (Xku∗, Yku∗) be the solution

to this problem, f∗

ku be the corresponding objective function value, and f1,ku∗

and f_2,ku∗ be values of functions f1 and f2, respectively. Let Eku be the error

rate for separating the sets Ak

u and Auk at iteration k over the set A, that is

Eku = |{a ∈ Ak u : ϕku(a) > 0} ∪ {b ∈ Aku : ϕku(b) < 0}| |A| , where ϕk_u(a) = max i∈Iku min j∈Jku i hxij∗, ai − yij∗
.

2: ( The first stopping criterion) If max{f_1,ku∗ , f_2,ku∗ } ≤ ε₁ then set Ak+1_u =

∅, Qk+1

u = Au\ Q

k

u and stop. (Xku∗, Yku∗) is the piecewise linear boundary for

set Au.

3: ( The second stopping criterion) If k ≥ 2 and f_k−1,u∗ − f_ku∗ ≤ ε₂ then set

Ak+1

u = ∅, Qk+1u = ∅ and stop. (Xku∗, Yku∗) where Xku∗ = Xk−1,u∗, Yku∗ =

Yk−1,u∗ is the piecewise linear boundary for set Au.

4: ( The third stopping criterion) If Eku ≤ ε3 then set Ak+1u = ∅, Qk+1u = Au\Q k u

and stop. (Xku∗_{, Y}

ku∗) is the piecewise linear boundary for set Au.

5: ( Refinement of sets of undetermined points) Compute

fku,min = min

a∈Ak

u

ϕk

u(a)

and the following set of easily classified points by function ϕk

u:

Qk+1_u =a ∈ Ak

Refine the set of undetermined points from the set Au as follows:

Ak+1_u = Ak_u\ Qk+1

u ,

6: ( Adding new hyperplanes)

1. If f∗

1,ku > ε1 then set

si_k+1,u = si_ku+ 1, J_ik+1,u = J_iku∪ {si_k+1,u}

for all i ∈ Iku. Set

xij = xi,j−1,∗, yij = yi,j−1,∗, i ∈ Iku, j = sik+1,u.

2. If f∗

2,ku > ε1 then set

rk+1,u = rku+ 1, Ik+1,u = Iku∪ {rk+1,u}, Jrk+1,uk+1,u = J

ku

rku.

Set

xij = xi−1,j,∗, yij = yi−1,j,∗, i = rk+1,u, j ∈ Jrkuku.

7: ( New starting point) Set

Xk+1,u _{= (X}ku∗_{, x}

ij, i ∈ Ik+1,u, j ∈ Jik+1,u),

Yk+1,u = (Yku∗, yij, i ∈ Ik+1,u, j ∈ Jik+1,u),

lk+1,u =

X

i∈Ik+1,u