Sample source selection

5.3 Hybrid query rewriting . . . 57

5.3.1 Global join decomposition . . . 61 5.3.2 Joins generation strategy . . . 61 5.3.3 Query rewriting algorithm . . . 63 5.3.4 Hybrid rewriting expressions evaluation . . . 69 5.4 Sub-queries sorting optimization . . . 72

5.4.1 Cost estimation algorithm . . . 72 5.4.2 Cost-based and shared links sorting approach . . . 74 5.5 Query evaluation. . . 76

5.5.1 Bindings strategy . . . 76 5.5.2 Parallelism strategy . . . 76 5.6 Conclusion . . . 77

5.1

Introduction

In the previous chapter, we have defined a semantics for federated SPARQL queries and specified it on top of standard SPARQL. Now, we aim at implementing a more expressive and efficient federated query processing engine in compliance with this semantics while taking into consideration the data distribution, data replication and results completeness challenges. As explained beforehand, SPARQL federated query engines intend to transparently integrate distributed and autonomous RDF data sources through their endpoints. The query processing is mainly performed with the following steps: (i) source selection, (ii) query rewriting, (iii) query planning and (iv) query evaluation.

In Chapter 3, we have seen that several engines, such as DARQ [64], FedX [75], SPLENDID [32], ANAPSID [2], LHD [89], WoDQA [4] and ADERIS [51], have been proposed. As a reminder, most of them use hybrid sources selection approach by combining metadata catalogs and sampling query-based approaches to get infor- mation on sources in order to accurately identify the relevant ones. Regarding the query rewriting step, Triple-based and BGP-based evaluations are respectively used for vertically and horizontally partitioned data. There are two main approaches for the query planning step, both based on heuristics. The first one uses heuristics with search algorithms (dynamic and greedy) and the last one applies heuristics on query patterns to sort them. Finally, several kinds of operators such as bind join, nested loop join and hash join are used to process the results during the query evaluation step.

In this chapter, we first introduce our sampling query-based source selection which aims at both identifying relevant sources and retrieving triple patterns cardinality. Then, the query rewriting problem is outlined and a query decomposition technique that aims at pushing as much as possible of the query processing towards the remote data sources for maximal filtering of the query results is proposed. Subsequently, we propose a query sorting approach based on cardinality cost estimation and shared variables heuristics. Finally, we introduce our redundancy aware query evaluation approach.

5.2

Hybrid sources selection

This section presents our source selection strategy to both identify relevant sources for candidate triple patterns and retrieve their cardinality statistics. As discussed in section 3.3.1.3, the metadata catalogs approach is not suitable for most of the data sources. Indeed, only few SPARQL endpoints provide them and they need to be

frequently updated to be accurate. Moreover, since we are in a context of dynamic data sources related to the nature of the Web, the sources undergo frequent changes. Thus, we decide to build our own data indexing through a Sampling Query-Based approach to acquire knowledge on data sources. More specifically, instead of only sending ASK queries to know whether a predicate appears in a given source or not, we propose to send a SELECT COUNT(*) query to retrieve at the same time an estimation of the cardinality of this predicate. In this way, we build two indexes:

idxPredicateSources and idxPredicateCardinality.

• The idxPredicateSources index records for each predicate the potential sources that are contributing to its results, in order to avoid unnecessary remote requests.

• The idxPredicateCardinality index provides cardinality statistics in order to estimate the triple patterns cost, which is useful during the query planning step. For each predicate the estimated cardinality will be the sum of the estimated cardinality in remote data sources (Card(p) = Σicard(p, si)).

The accuracy of this cardinality is a key factor of our optimization since our query planning approach is partially based on it. Thus, some less selective predicates such as rdf:type and owl:sameAs or non-selective predicates (variable as predicate for instance) need more processing. Indeed, the SELECT COUNT query will tend to give a high number of cardinality due to their wide use in data sources or because the predicate is a variable. Therefore, instead of sending a query with only the predicate and variables as subject and object, we replace the subject and object by their values when they are known to refine the cardinality estimation and possibly avoid source selection overestimation.

When the subject, the object and the predicate of a triple pattern are variables (?s

?p ?o), we assign the M ax_Card as cardinality to this triple pattern. M axCard is a

high number arbitrarily fixed to denote the maximum cardinality. For the sake of simplicity this M axCardvalue is also assigned to predicates which are property paths

operator such as the binary operator ZeroOrMorePath (*). Indeed, estimating the cardinality of these kind of predicates can be very time-consuming. Therefore, a trade-off between the cardinality estimation accuracy and the processing time needs to be made.

5.2.1

Sample source selection

The following paragraph illustrates the source selection approach through an exam- ple of federated query (Q5.1). This query retrieves the name and total population of

each departement and is evaluated over three remote data sources. The data sources

S1 and S2 contain French geographic [28] data and the data source S3 contains

French demographic [27] data. These RDF graphs are published by the National Institute of Statistical and Economical Studies (INSEE).

Query Example 5.1: federated SPARQL query over three data sources 1 p r e f i x d e m o : < h t t p :// rdf . i n s e e . fr / def / d e m o # > 2 p r e f i x geo : < h t t p :// rdf . i n s e e . fr / def / geo # > 3 s e l e c t ? n a m e ? t o t a l P o p w h e r e { 4 ? r e g i o n geo : s u b d i v i s i o n D i r e c t e ? dpt . 5 ? r e g i o n geo : c o d e R e g i o n ? v . 6 ? dpt geo : nom ? n a m e . 7 ? dpt d e m o : p o p u l a t i o n ? p o p L e g . 8 ? p o p L e g d e m o : p o p u l a t i o n T o t a l e ? t o t a l P o p . 9 }

This query involves 5 triple patterns (from line 4 to line 8). The predicates (geo:subdivisionDirecte, geo:codeRegion and geo:nom) related to geographic data belong to S1and S2and the predicates (demo:population and demo:populationTotale)

related to demographic data belong to S3.

Thus, the previous source selection approach applied to Q5.1 will generate the following indexes: • idxPredicateSources = { geo : subdivisionDirecte → {S1, S2}; geo : codeRegion → {S1, S2}; geo : nom → {S1, S2}; demo : population → {S3};

demo : populationT otale → {S3}}

• idxPredicateCardinality = {

geo : subdivisionDirecte → 3934; geo : codeRegion → 27;

geo : nom → 41458;

demo : population → 37149;

demo : populationT otale → 37147}

The index idxPredicateSources prevents from sending the triple patterns relevant for the source S1 and S2 (line 4 to line 6) to the source S3, and conversely, the triple

patterns relevant for S3(line 6 and line 7) to S1and S2.

The index idxPredicateCardinality, for instance, informs that the first triple pattern (line 4) is less selective than the second triple pattern (line 5) based on their cardinality. Therefore, processing the second triple pattern before and propagating afterwards the values of the variable ?region while processing the first triple pattern will be more efficient.