The algorithm

The learning algorithm is essentially a top-down learning approach combined with a reduction task. The top-down step is used to solve the sub-problems of the given learning problem and the reduction is used to reduce and combine the sub-solutions

Figure 6.4: Top-down learning in SPaCEL with multiple workers. U_ِ(TOP) Uِ(C1) TOP C1 C4 C11 C12 Uِ(C4) worker 1 worker 1 worker 1 worker 2 p1 p2 p3 p4 p5 p6 . . . worker 2 C3 . . . C41 C2 positive examples n1 n2 n3 n4 n5 n6 . . . negative examples

into an overall solution. The top-down step is performed by the downward refinement operator and a combination strategy while the reduction step currently uses a set coverage algorithm to choose the best partial definitions and disjunction to form the overall solution.

Algorithm 6.1 describes the main part of our learning algorithm, the reducer. It

chooses the best concepts (i.e. highest score, based on the search heuristic described in

Definition 5.4) from the search tree and uses theSpecialisealgorithm (see Algorithm

6.2) for refinement and evaluation until the completeness of the partial definitions is sufficient. Concepts are scored using an expansion heuristic that is mainly based on the correctness of the concepts. In addition, a penalty is applied for complexity of the concepts (short expressions are preferred), and bonuses for accuracy and accuracy gained (see Definition 5.4).

The sets of new descriptions, partial definitions and counter-partial definitions re-

turned from the specialisation algorithm are used to update the corresponding data

structures and the set of covered positive and negative examples in the learning algo- rithm. In addition, the new descriptions are combined with the counter-partial defi- nitions to create new partial definitions if possible. It is important to note that the concepts that have been refined can be scheduled for further refinements.

The refinement operator in Definition 6.3 is infinite, but in practice each refinement step is finite, since it is only allowed to generate descriptions at a given length. For

Algorithm 6.1: Symmetric Class Expression Learning Algorithm– SPaCEL(K,E+,E−, ε)

Input: background knowledgeK, a set of positive E+ and negativeE−

examples, and a noise valueε∈[0,1] (0 means no noise)

Output: a definitionC such that|cover(K, C,E+)| ≥(|E+| ×(1−ε)) and cover(K, C,E−) =∅

1 begin

2 initialise the search treeST ={ } /* : TOP concept in DL */

3 cum pdefs =∅ (empty set) /* set of cumulative partial definitions */ 4 cum cpdefs = ∅ /* set of cumulative counter-partial definitions */ 5 cum cp=∅ /* set of cumulative covered positive examples */ 6 cum cn=∅ /* set of cumulative covered negative examples */

/* there may be more conditions, e.g.timeout */ 7 while |cum cp|<(|E+| ×(1−ε)) and|cum cn|<|E−|do

8 get the best concept B and remove it fromST /* see text */

9 (pdefs,cpdefs, descriptions) =Specialise(B,E+,E−) _{/* cf.Alg.6.2 */} 10 cum pdefs=cum pdefs ∪pdefs

11 cum cpdefs=cum cpdefs ∪cpdefs

12 cum cp=cum cp∪ {e|e∈cover(K, P,E+), P ∈pdefs} 13 cum cn=cum cn∪ {e|e∈cover(K, P,E−),P∈cpdefs}

/* online combination, see Sec.6.2.4 */ 14 foreach D ∈ descriptions do

/* if D can be ‘‘corrected’’ by existing cpdefs */ 15 if (cover(K, D,E−) \ cum cn) =∅ then

/* combine D with cpdefs if possible */

16 candidates=Combine(D,cum cpdefs,E−) _{/* cf.Alg.6.3 */} 17 new pdef=D ¬(_{A∈candidates}(A)) /* new partial def. */

18 cum pdefs=cum pdefs ∪new pdef

19 cum cp=cum cp∪cover(K, D,E+)

20 else

21 ST =ST ∪ {D}

/* explore more partial definitions to meet the completeness */ 22 if |cum cp|<(|E+| ×(1−ε))then

23 foreach D∈ST do

24 candidates=Combine(D,cum cpdefs,E−) 25 new pdef=D ¬(_{A∈candidates}(A))

26 cum pdefs=cum pdefs ∪new pdef

and later, when it is revisited, to concepts of length (N + 2), etc. For the sake of

simplicity, we useρ in the algorithms to refer to one refinement step rather than the

entire refinement. This technique is used in DL-Learner and discussed in detail in [82]. When the algorithm reaches a sufficient degree of completeness, it stops and reduces

the partial definitions to remove the redundancies using the Reduce function, which

is essentially a set coverage algorithm: given a set of partial definitions X and a set

of positive examples E+, it finds a subset X ⊆X such thatE+ ⊆_D∈X(cover(K, D,

E+_{)). The solution returned by the algorithm is a disjunction of the reduced partial}

definitions. However, returning the result as a set of partial definitions instead may be useful in some contexts, e.g. to make the result more readable. The reduction algo- rithm may be tailored to meet particular requirements such as the shortest definition or the least number of partial definitions. Note that the combination of descriptions and counter-partial definitions in Algorithm 6.1 is one of the combination strategies

implemented in our evaluation. This strategy is called anon-the-fly combination strat-

egy; it gave the best performance in our evaluation (see Section 6.3.1). A discussion of the combination strategies is given in Section 6.2.4.

The specialisation is described in Algorithm 6.2. The specialisation performs the refinement and evaluation of the concepts assigned by the learning algorithm. Firstly, it

refines the given concept (ρ(C)) and evaluates the result (cover(K, C,E+) andcover(K,

C,E−)). Irrelevant concepts are removed from the result as no partial definition or

counter-partial definition can be computed though the irrelevant concept specification. Then, the specialisation finds new partial definitions, counter-partial definitions and descriptions from the refinements. Practically, redundancies are often checked before evaluating descriptions to avoid redundant evaluations and duplicated descriptions in the search tree as a description can be generated from different branches.

Algorithm 6.3 describes the combination algorithm that is used to combine the

descriptions and counter-partial definitions to construct new partial definitions. This is basically a set coverage algorithm. A smallest set of counter-partial definitions that together cover all negative examples covered by the given expression will be returned. This set of counter-partial definitions is then used to correct the given expression.

Algorithm 6.2: Specialisation algorithm –Specialise(C,K,E+,E−) Input: a descriptionC and a learning problem LP =K,(E+,E−).

Output: a triple consisting of a set of partial definitionspdefs ⊆ρ(C); a set of

counter-partial definitions cpdefs ⊆ρ(C); and a set of new

descriptionsdescriptions ⊆ρ(C) such that

∀D∈descriptions:Dis not irrelevant andD /∈(pdefs∪cpdefs), in

which ρ is the refinement operator defined in Definition 6.3.

1 begin

2 pdefs ={D∈ρ(C) |cover(K, D,E+)=∅ ∧ cover(K, D,E−) =∅} 3 cpdefs ={D∈ρ(C) |cover(K, D,E+) =∅ ∧ cover(K, D,E−)=∅} 4 descriptions ={D∈ρ(C) |cover(K, D,E+)=∅ ∧ cover(K, D,E−)=∅} 5 return (pdefs, cpdefs, descriptions)

Algorithm 6.3: Combination algorithm – Combine(C,cpdefs,E−)

Input: a description C, a set of counter-partial definitionscpdefs and a set of negative examplesE−

Output: a setcandidates ⊆cpdefs such that

cover(K, C,E−)⊆_{P∈candidates}(cover(K, P,E−)) 1 begin

2 candidates =∅ /* candidate counter-partial definitions */

3 cn c=cover(K, C,E−) /* negative examples covered by C */ 4 while cpdefs=∅ and cn c=∅do

5 sort cpdefs by descending coverage of negative examples

6 get and remove the top counter-partial definition Dfrom cpdefs

7 if (cover(K, D,E−)∩cn c)=∅then 8 candidates=candidates∪D 9 cn c=cn c\ cover(K, D,E−)) 10 if cn c=∅ then

11 return ∅ /* return an empty set */

12 else

13 return candidates 14