Evaluation of ABox Update Implementation

In this section we provide some empirical results wrt. the efficacy of the various optimization techniques introduced above.

For the evaluation we use our own implementation of logical ABox update in ECLiPSe-Prolog [ECLiPSe Implementors Group, 2009], and an implementation of projective ABox updates (cf. section 2.4.2) by Hongkai Liu in Java.7 _{The testing} was done in joint work with Hongkai Liu, who kindly also provided the testing data. Testing was done using a set of 2000 random ALC ABoxes containing between two and twenty-three assertions, and a set of 2000 random, unconditional updates con- taining between one and eleven effects. The methodology underlying the generation of the random input is described in [Liu, 2009].

To begin with, figure 5.2 (courtesy of Hongkai Liu) shows the sizes of updates based on projective updates (plotted as ×) and a naive implementation of logical updates in ALCO@ (plotted as +), where by a naive implementation of logical updates we refer to an implementation that is directly based on the construction of logical updates as recapitulated in section 2.4.2 — with no optimizations whatsoever. The horizontal axis shows the size of the input ABox, and the vertical axis shows the logarithmized size of the output ABox (ln(n)). Figure 5.2 seems to confirm the theoretical expectation that projective updates (being polynomial in the input) behave much better than logical updates, which are exponential in all of the input. Let us next discuss the efficacy of the various optimization techniques for logical updates introduced above.

We start with an observation on updating ABoxes to either CNF or DNF. The other optimizations notwithstanding, updating to CNF due to the duplication of identical assertions results in updated ABoxes that are significantly bigger than those obtained by updating to DNF. As expected in the experiments the CNF rep- resentation proved far superior.

For all the other optimization techniques the bottom-line is that the syntactic variants outperform their counterparts involving reasoning. For example, in the 2000 random input ABoxes, together with the 2000 random updates we could not find a single assertion that was independent from the update by resorting to reasoning for which we could not detect this syntactically. The costs for the syntactic fragments of the optimization techniques proved to be negligible, and on the random input data all of the optimization resulted in smaller updated ABoxes. Hence, our implementation syntactically does the following:

• Transform ABoxes into a certain normal form; • Exploit the unique name assumption;

7

For clarity, in this section we refer to ABox update as logical update to distinguish it from projective update.

5.2 Implementing ALPs atop ABox Update

Size of Updated ABoxes

0 5 10 15 20 25 30 0 50 100 150 200 250 300 + : Logical updates in ALCO@

× : Projective updates

The numbers in vertical axes are logarithmized (ln(n)). Figure 5.2: Comparing the size of projective and logical updates

• Update directly to negation normal form; • Detect subsuming disjuncts; and

• Identify assertions that are independent from the update.

Regarding optimizations that require reasoning we could make the following ob- servations:

• The costs for detecting and removing entailed assertions were prohibitively high. We have not looked into heuristics for identifying suitable candidate assertions, though.

• With regard to the identification of determinate updates we could make an interesting observation: On the set of the 2000 random input data the syntactic technique for identifying subsuming disjuncts was able to detect virtually all determinate updates.8

8

This means that there is no assertion α in the testing data such that a concept or role name occurs both negatively and positively in α.

Summing up, neither of the two optimization ideas based on reasoning proved worth the cost.

With this we come to the comparison of our optimized implementation of logi- cal updates to the implementation of projective updates on the same 2000 testing data. In figure 5.3 (also courtesy of Hongkai Liu) we can see that the optimized implementation works well on the random testing data. The left graph shows the size of the updated ABoxes for all 2000 testing data, while the graph on the right shows the respective sizes of those updated ABoxes that are of size up to 400 000. The sizes of projective updates are plotted as ×, whereas • and +denote the sizes of logical updates in ALCO+ and ALCO@, respectively. Here, the horizontal axis shows the size of the input ABox, and the vertical axis shows the logarithmized size of the output ABox in the left graph, and the logarithmized run-time in milliseconds in the right graph — both times (ln(n)). We do not show the computation time for updated ABoxes in ALCO+ (•) in the right graph — it is almost identical to that of projective updates (×).

Size of Updated ABoxes Time for Updating

0 2 4 6 8 10 12 14 16 18 0 100 200 300 400 500 2 3 4 5 6 7 8 9 10 0 100 200 300 400 500

+ : Logical updates in ALCO@ × : Projective updates

• : Logical updates in ALCO+

The numbers in vertical axes are logarithmized (ln(n)). Figure 5.3: The size of projective and optimized logical updates

We can make the following observations: In general, the logical updates in ALCO+ are the smallest. Next the logical updates in ALCO@ frequently behave better than the projective updates — however, there is a number of cases where their theoretical inferiority shows up. Overall, the behavior of projective updates is more predictable

5.2 Implementing ALPs atop ABox Update

than that of logical updates in ALCO@. For the latter, all observed “bad” cases occurred if the update contained assertions that concerned subconcepts and -roles that occurred at a quantifier depth greater than one in the input ABox — i.e. the update concerned unnamed individuals.

The following numbers also highlight the differences between the competing rep- resentations: The random input data consume about 17 MB of disk space. The updated ABoxes consume about 20 MB in ALCO+, about 1.1 GB in ALCO@, and about 1.5 GB as projective updates. To be fair, in ALCO@ not even all updated ABoxes could be computed — there are about sixty ABoxes that are not included in the 1.1 GB but in the 1.5 GB of the projective updates. However, the most striking fact is the superiority of ALCO+.

This somewhat surprising superiority of the exponential logical updates in ALCO+ over the polynomial projective updates can at least in part be attributed to the fact that none of the optimization techniques introduced above can directly be applied to projective updates. Developing powerful optimization techniques for projective updates is thus an interesting topic for future work.

Let us also briefly comment on the issue of iterated updates: The results in figures 5.2 and 5.3 are for non-iterative updates. As discussed in [Liu, 2009] projective updates do not handle iterated updates well in practice. One way around this is to merge iterated updates into a single one [Liu, 2009], e.g., to merge the updates U1= {A(a)} and U2 = {¬A(a)} into U = {¬A(a)}. While for projective updates this technique is vital, for ALCO+ it is not yet clear whether it is cheaper to repeatedly enforce the same effect or to accept a small increase in redundancy of the updated ABox.

Summing up, as of now logical updates in ALCO+ appear to be the best repre- sentation formalism for ABox update.