Data Flow

In this section, we lay down the general framework of the MultiBlock ap- proach, which is independent of a specific similarity measure or aggregation. At the same time, we define which properties a similarity measure or aggre- gation must adhere to in order to be used in our approach. The subsequent section will specify various similarity measures and aggregations that can be

plugged into the framework in order to provide a complete indexing method. MultiBlock is organized in two phases: At first, for each distance measure, an index is generated. Afterwards, all distance values are aggregated into a single multidimensional index.

The indexing phases are executed following the components of the linkage rule. Analogous to the computation of the similarity of two entities, the index of a single entity is computed by successively applying the components of the given linkage rule. Table 5.1 summarizes how computing the similarity between two entities compares to computing the index of a single entity.

Phase

Step Compute Similarity Compute Index Input Two entities A single entity Retrieve Retrieve property paths of the given entity(s) Transform Transform values

Compare Similarity of two values Index one value Aggregate Aggregate similarity scores Aggregate indexes

Table 5.1: Steps involved in computing the similarity of two entities compared to computing the index of a single entity.

Index Generation

For each similarity measure in the linkage rule, an index is built, which consists of a set of vectors that define locations in the Euclidean space. The basic idea of the indexing method is that it preserves the distances of the entities, i.e., similar entities will be located near each other in the index. The index generation is not restricted to a specific distance measure.

In order to be used for MultiBlock, for each distance measure must define an additional indexing function. The indexing function has to be defined in addition to the distance function been introduced in Section 2.5.

In total, for each distance measure, two functions have to be defined: • The distance function:

fd _{: Φ × Φ → R} (5.1)

• While the distance function computes the distance between a pair of values, the indexing function generates an index for a single entity:

where P denotes the power set (i.e., it might generate multiple indices for a single entity) and n is the dimension of the index. The first argument denotes the entity to be indexed, which may be either in the source or target set. The second argument denotes the similarity threshold. index includes two modifications of the standard block func- tion presented in the preliminaries. Firstly, it does not map each entity to a one-dimensional block, but to a multi-dimensional index. This way increases the efficiency as the entities are distributed in multiple dimensions. Secondly, it does not map each entity to a single index, but to multiple indices at once, similar to multi-pass blocking. This avoids losing recall if an entity cannot be mapped to a definite index, such as in some string similarity measures (see Section 5.3.2).

The index function must adhere to the property that two entities whose distance according to fd _{is below the threshold must share a block. More}

formally, given two values v1, v2 and a threshold θ, fd and index must be

related by the following equivalence:

fd(v1, v2) ≤ θ ⇐⇒ index(v1) ∩ index(v2) 6= ∅ (5.3)

Section 5.3.2 gives an overview over the most common distance measures that can be used in MultiBlock.

Index Aggregation

In the index aggregation phase, all indexes that have been built in the index generation phase are aggregated into one compound index. The aggregation function preserves the property of the index that two entities within a given distance share the same index vector. Generally, aggregating the indexes of multiple similarity measures will lead to an increase in dimensionality, but the concrete aggregation function depends on the specific aggregation type. For instance, when aggregating a 2-dimensional geographic index and an 1-dimensional string index using an average aggregation, the resulting index will be 3-dimensional. Section 5.3.3 outlines the concrete aggregation functions for the most common aggregation types.

In order to be used for MultiBlock, each aggregation must define the following functions:

• A function that aggregates multiple similarity values:

where n is the number of operators to be aggregated. The first argu- ment denotes the similarity values to be aggregated. As an aggrega- tion may weight the results of the underlying operators, such as the weighted average aggregation, the second argument denotes the weight of the specific operator. This function has already been introduced in Section 2.5.

• A function that aggregates multiple s:

aggIndex : P(Nn) × P(Nn) → P(Nn) (5.5) where P denotes the power set and n is the dimension of the index. Note that while aggIndexa only aggregates two sets of indices at once,

it can also be used to aggregate multiple sets by calling it repeatedly. • A function that updates the threshold of the underlying operators in

order to retain the condition that two entities within the threshold share an index.

modif yT hreshold : R × N → R (5.6) The first argument denotes the threshold on the aggregation. As in the similarity aggregation function, the second argument denotes the weight of the specific operator.