Linus

Linus was first presented in detail by Lavraˇc [75] in 1990. It generalized ideas from Bratko and colleagues [18] that were applied for expert system design in a medical domain, especially the algorithm QuMAS (Qualitative Model Acquisition System). An introductory presentation of Linus was provided by Lavraˇc and Dˇzeroski in Section 5.4 of their book on ILP [76]. In the following, relevant parts of that presentation are adopted in the sense of our framework.

Examples in E⁺ and E⁻ are ground facts and may contain structured, but non-recursive terms. As such, they are DHDB clauses, cf. Definition 5. The training examples provide an extensional definition of the target predicate p/a.

The learning task is to find an intensional definition of this predicate that can be applied to unseen instances. Ultimately, this corresponds to the learning task as defined in our framework.

Background knowledge B can have the form of DDB clauses, cf. Definition 4.

It may be non-ground, i. e. intensional, and possibly recursive. Furthermore, a symmetric predicate equals =/2 is a built-in predicate in Linus and applicable to

3.2. TRADITIONAL APPROACHES TO PROPOSITIONALIZATION 37

variables of the same type. It adds both expressivity and complexity. According to tasks described in the ILP book [76], background knowledge is frequently in the form of ground facts and non-structured. The same applies to examples. This form is also suggested as advantageous within our framework.

Clauses for the clause set C in the sense of our framework take the form p(X₁, ..., X_a) : − < Atom > . (3.10) where < Atom > can be one of the following

1. a positive constrained literal, cf. Definition 6, e. g. q(Xi, Xj) or Xi = Xj

with 1 ≤ i, j ≤ a, or

2. a positive determinate literal, cf. Definition 7, e. g. r(Xi, Xj, Y ) with 1 ≤ i, j ≤ a where Y has a unique binding given bindings of X_i and X_j.

Clause generation for C is typically exhaustive. That means, all possible ap-plications of background predicates on the arguments of the target relation are computed, considering types. Each such possible application results in an at-tribute. For determinate body literals with more than one new variable, the cor-responding number of new attributes is produced. Tuple construction is achieved by calling the corresponding predicates for each target relation tuple.

If a call of a constrained clause C ∈ C succeeds for an example, the corre-sponding feature value is set to true, else to f alse. This can be seen as an ex-istential feature in the sense of the framework presented above. More elaborate, val(C, e, B) contains at most one element, because of the usage of a constrained clause. With C and val(C, e, B) as argument, ϕ∃ is applied as presented in our framework.

If a call of a determinate clause C ∈ C succeeds for an example, the corre-sponding feature value(s) is / are set to the values of the new variable(s) of the body literal. More elaborate, val(C, e, B) contains at most one element, because of determinacy. The propositionalization function used here is identity ϕid, i. e.

values of new variables in determinate literals are directly used as new attribute values.

After propositional rule or tree learning, learning results are converted back into a first-order theory. Clauses building the final Linus theories have again p(X₁, ..., X_a) as head. The body of a clause in a hypothesis is a conjunction of literals that can take the following forms.

1. a binding of a variable to a value, e. g. X_i = x with 1 ≤ i ≤ a,

2. an equality of pairs of variables occurring in the head of the clause, e. g.

X_i = X_j with 1 ≤ i, j ≤ a,

3. a positive constrained literal, e. g. q(Xi, Xj) with 1 ≤ i, j ≤ a, and

4. a positive determinate literal and further literals to bind its new variables, e. g. f (Xi, Xj, Y ), Y = y with 1 ≤ i, j ≤ a.

Thus, hypotheses take the form of constrained DHDB clauses. For (1) and numeric X, X_i > x and X_i < x are also allowed, analogous for (4) and numeric y. For guidance of induction by reducing the search space, any subset of the four cases can be excluded. For instance, focusing on (1) finally yields an attribute-value language, i. e. learning from the target relation only. So, the language bias in Linus is declarative.

In the literature, there is often a restriction to (2) and (3), i. e. function-free clauses, which causes propositionalization results to be based on constrained clauses only, and hence such a table is completely Boolean.

Post-processing within Linus especially involves irrelevant literal elimination.

The treatment of irrelevancy is dealt with in this dissertation below.

We now illustrate with an example, which is adopted from Section 5.3.2 in the ILP book [76].

Example 8 The target relation is daughter(X,Y) and means that person X is the daughter of person Y. The task is to define the target relation with the help of the background knowledge relations female, male, and parent. All variables are of type person = {ann, eve, pat, sue, tom}. Figure 3.1 shows the input data.

Training examples

daughter(sue,eve). pos daughter(ann,pat). pos daughter(tom,ann). neg daughter(eve,ann). neg

male(pat).

male(tom).

female(ann).

female(sue).

female(eve).

parent(eve,sue).

parent(ann,tom).

parent(pat,ann).

parent(tom,sue).

Background knowledge

Figure 3.1: A daughter family relationship problem in Prolog form C contains the following clauses, here excluding =/2:

1. daughter(X,Y) :- female(X).

2. daughter(X,Y) :- female(Y).

3. daughter(X,Y) :- male(X).

4. daughter(X,Y) :- male(Y).

5. daughter(X,Y) :- parent(X,X).

6. daughter(X,Y) :- parent(X,Y).

7. daughter(X,Y) :- parent(Y,X).

3.2. TRADITIONAL APPROACHES TO PROPOSITIONALIZATION 39

8. daughter(X,Y) :- parent(Y,Y).

Figure 3.2 shows the results of the application of the background knowledge predicates in the form of the clauses in C, with f for female, m for male, and p for parent.

Figure 3.2: Propositional form of the daughter relationship problem (1 for true, 0 for f alse)

Note that there are examples of redundant features given here, e. g. male because it is the complement of female, or parent(X,X) because the set of its pos-sible values is a singleton that cannot differentiate between positive and negative examples.

From the propositional representation, an attribute value learner may induce a rule such as the following:

if [female(X) = 1] and [parent(Y,X) = 1] then class = pos

This can be transformed into the following DHDB clause as the output of Linus:

daughter(X,Y) :- female(X), parent(Y,X).

To demonstrate limitations of the approach, we investigate its application to our running example.

Example 9 Here, we assume that only two types numeric and nominal are de-fined for the running example database. Table T is here referred to by the predicate symbol t. It has three numeric arguments in the first argument positions and one nominal argument in the last argument position.

C would then contain the following constrained clauses:

t(W,X,Y,Z) :- a(W,W,W,W,Z).

t(W,X,Y,Z) :- a(W,W,W,X,Z).

t(W,X,Y,Z) :- a(W,W,W,Y,Z).

t(W,X,Y,Z) :- a(W,W,X,W,Z).

t(W,X,Y,Z) :- a(W,W,X,X,Z).

...

t(W,X,Y,Z) :- a(Y,Y,Y,Y,Z).

...

t(W,X,Y,Z) :- b(W,W,W,Z).

...

In the first clause, the variable W for the first numeric argument of t is used to fill all argument places for a, where numeric values are needed. For the nominal arguments — the last in both relations — the same variable must be shared, here Z. In the following clauses, other combinations of shared variables are used.

These clauses are semantically questionable because there are unifications of unrelated arguments, e. g. for the first clause, the fourth argument of relation a is a descriptive attribute of objects of kind a, which has nothing to do with the identifier attribute of relation t in t’s first argument position, but the literals for t and a share a variable in the corresponding argument positions.

Of course, we could define types differently, e. g. based on the names of at-tributes. In this case, C would be empty since each non-target relation encom-passes at least one attribute not contained in the target relation. Thus, no variable could be shared here between head and body of a clause. Overall, the application of Linus seems not appropriate here.

We will return to the case of determinate clauses in C in the context of Dinus.

To finish this section, we investigate complexity issues of Linus. This com-plexity originates strongly from the usage of variable permutations in the con-strained clauses of C. According to Section 5.5 of the ILP book [76] relevant aspects are the following. Given

• u as the number of distinct types of arguments of the target predicate p

• ui as the number of distinct types of arguments of a background predicate qi

• n_i,s as the number of arguments of qi of a type Ts

• k_ArgTs as the number of arguments of p of type T_s

Then, the number of new attributes derived from qi is computed by the fol-lowing formula:

kN ew,qi =

ui

Y

s=1

(kArgTs)^ni,s (3.11)

This means that the ni,s places for arguments of qi of type Ts can be filled in (kArgTs)^ni,s ways independently from choosing the arguments of qi which are of different types.

3.2. TRADITIONAL APPROACHES TO PROPOSITIONALIZATION 41

We point to the circumstance that results are the same for the usage of de-terminate literals in C, considering old variables only. New variables have no significant influence on the complexity of C generation. The same names of new variables can be used in clauses that differ in their usage of old variables only.

If the built-in background predicate for equality =/2 is used, its symmetry is important and thus the number of possible applications amounts to:

kN ew,==

Sums over such terms for all background knowledge predicates, i. e. for all clauses C ∈ C lead to the final number of newly derived attributes, which is exponential in the number of body literal arguments of the same type. This demonstrates aspects of the complexity of the generation of clauses for C.

The application of clauses C ∈ C is less complex. Before a constrained clause is applied for a certain example e, all variables in the clause are instantiated. The body literal L of C thus becomes ground. It remains to be checked if B contains L. For a clause C known to be determinate, the first matching fact in B for the body literal of C can be found with the same complexity, which is polynomial in the number of old variables in L.