Near Optimal Document Exchange

As we have already seen, many of the distances of interest are metrics. Much study has been made of metric spaces, and if we know that a distance is a metric this enables us to tell a lot about its structure. In fact, all of the metrics that we study here are defined in a particular way: we define a series of unit cost operations (such as the edit operations), and set the distance between two objects as the minimum cost sequence of operations to turn one into the other. They are therefore editing distances in the sense of Definition 1.1.1. In this situation, we can further exploit properties of the metric, and come up with a communication scheme that allows document exchange in at most twice the optimal amount of communication. We begin this section with a description of how this can be achieved with a known bound on the editing distance, and then go on to describe a multi-round adaptation that does not require an a priori bound on the distance.

6.3.1

Single Round Protocol

LetRbe a symmetric relation on a setX. Letd(x, y)be an editing distance on this set defined as usual from the transitive closure ofR— that is,d(x, y)is the minimumnsuch thatRn_{(x, y). Suppose that}

there are two individuals, A who holdsx∈X and B who holdsy∈X, such thatd(x, y)≤lfor some knownl. The range ofdis the non-negative integers, and we will refer to this as the distance. LetG_ibe the undirected graph whose vertices areX and whose edges are{(x, y)|d(x, y)≤i}.

Lemma 6.3.1 deg(G2l)≤(deg(Gl))2.

Proof. deg(G2l)corresponds to the greatest number of vertices at a distance at most2l (in the graph

corresponding toR) from any particular vertex,x. As the metric is defined by unit cost operations, these vertices will be those which are at mostl from some vertex at mostl fromx. Since from each vertex there are at mostdeg(G_l)vertices at distance at mostl, the lemma follows immediately. ✷ Following the model of Orlitsky, we shall temporarily employ a hypergraphH whose vertices are the members of X and whose hyperedges are ey = {x : d(x, y) ≤ l}. Both parties can calculate

this hypergraph independently and without communication (if some parameter of the graph is not implicit, A can prepend this information to the message with small additive cost). They can then use a deterministic scheme to assign colours to the nodes of the hypergraph with some number of colours. We useχ(H)to denote the chromatic number ofH, the smallest number of colours required to colourH in order to ensure that for every hyperedge, all nodes in that edge have a unique colour. A’s message is the colour ofx. By the colouring scheme and the specification of the problem, B knows that

xmust be amongst the vertices iney, and can use the colour sent by A to identify exactly which it is.

We can place upper and lower bounds on the single message complexity by considering this scheme. Since we must be able to distinguishxamong the vertices in the hyperedge x, at leastlog(deg(H))

bits must be sent overall. To send the colour of x, we need logχ(H) bits [KN97]. Let S be the number of bits necessary in a single message to allow B to calculatex(the single message cost). So

Lemma 6.3.2 χ(H)≤deg(G2l).

Proof. χ(H)is the number of colours required to colour the vertices ofH such that any two vertices which are both at most distancelfrom any pointxare coloured differently. Considery andy such thatd(x, y) ≤ l, d(x, y) ≤l. Sincedis a metric, d(y, y)≤ d(y, x) +d(x, y) = d(x, y) +d(x, y) ≤ 2l.

Any colouring which ensures that any two points within distance2lhave different colours is therefore a colouring of the hypergraphH. A colouring ofG2lachieves this, showing thatχ(H) ≤χ(G2l). On

the assumption thatG2lis not complete, is connected and has degree of three or more (which will be

satisfied by the examples in which we are interested), thenχ(G2l)≤deg(G2l), [Gib85] and so the lemma

follows. ✷

Theorem 6.3.1 log(deg(G_l))≤S≤ 2 log(deg(G_l)).

Proof. It is clear thatdeg(G_l) = deg(H); in both cases the degree of each vertex corresponds to the number of vertices a distance at mostlaway. We combine this with Lemma 6.3.2 and Lemma 6.3.1 to get the fact thatlog(deg(H))≤S≤ log(χ(H))to prove the theorem. ✷ Thus, under certain conditions, a single message can be at worst twice the lower bound on its length for any number of messages. A similar theoretical result to that shown here is given by Orlitsky [Orl91]. By considering a more restricted class of functions, we can construct a concrete protocol which achieves the above bound. Note that our upper bound is related to the degree of a graph which we can calculate, and algorithms exist which will colour a graph with this number of colours; hence it is achievable in practice. Tractable instances of this technique are discussed further in Section 6.3.3.

Multi-round versions of the Colouring Protocol

The above section assumes that we know a tight upper bound,l, on the distance between the objects in question. This is an assumption shared by all previous works on this subject. In earlier sections, we have shown how to arrive at an approximate upper bound on distance for most distances, but a few are not covered. Here, we briefly discuss how to adapt the above technique to work in the absence of an upper bound on the distance, by introducing a probabilistic test. The modified protocol works in

O(logd)rounds, instead of a single message, wheredis the true distance. In the first round the users run the protocol with the assumption thatd(a, b)is upper bounded by some small constant. B then uses a hash function on the value ofagenerated by this protocol, and sends it to A. In turn, A computes the same hash function on the true value ofa, and compares the two. If they agree, then it is assumed that the two values ofaare the same, and the protocol terminates. We choose our hash functions in order to ensure that the probability of this happening whenaandbdo not in fact agree is small. The number of bits to represent this hash function is small in comparison to the number of bits needed to represent the messages exchanged, which isO(logn/δ). If not, they double the distance, and repeat the protocol, until they agree. This requires2 logdrounds, and in the worst case transmits four times as many bits as if an exact value fordwas available in advance.

6.3.2

Application to distances of interest

We will now show how Theorem 6.3.1 can be applied to the distances of interest. We shall make use of the definition of a non-trivial communication scheme (Definition 6.1.2).

Hamming distance is a metric on the space of strings of lengthnover an alphabetσ, and is induced by defining a symmetric relation on strings which differ in a single character. It is therefore an editing distance. Forl < n/2,G2lis not complete, and for any non-trivialn,G2lwill have degree higher than

two: deg(Gl) = l

consider the upper bound on this quantity for up tolHamming differences described in Lemma 6.2.1. This usesllog((|σ| −1)(n/l)) + 2 bits, hence we have the result that up tol Hamming errors can be corrected using a single message with asymptotic cost at most2l(log((|σ| −1)(n/l)) + 2), which is non- trivial forl < n/2.

Edit distance is defined by charging unit cost to insertions, deletions or alterations of a character. Again, forl < n/2(ndenotes the length of the longer string),G2lis not complete and will have degree

more than two. Also, there is no concise exact expression for the degree ofG_l, but we can use the binary encoding (Lemma 6.2.2) for an upper bound onlog(deg(Gl)). We can encode up toeedit operations

using at moste(log 2(|σ|(n+ 1)/e)) + (elog logn)bits, so2 log deg(Gl)≤2l(log(|σ|(n+ 1)/l) + log logn)

which is non-trivial forl < n/2.

Edit Distance with moves is an editing distance. Using the bitwise encoding, this gives an upper bound on the single message cost of6dlogn.

Compression Edit Distance is an editing distance. Using the simple encoding from Lemma 6.2.3 we can achieve a single message cost of18llog|a|bits, non-trivial forl≤_{18 log}|a|log_|a||₊σ|_|b|.

Permutation Edit Distance has only a single operation, of moving a single element. We can use the bitwise encoding to get a single message cost of4llognbits, non-trivial forl < n/4.

Reversal Distance is found from unit cost reversals. Following the same argument, we get a single message cost of4llognbits, non-trivial forl < n/4.

Transposition Distance is similar to reversal distance, and yields a single message cost of6llogn

bits, non-trivial forl < n/6.

Swap Distance is similar to Permutation Edit Distance, and can have a single message cost of

2llognbits, non-trivial forl < n/2.

RITE Distances allow combinations of the various operations, reversals, indels, transpositions and edits, each at unit cost. By using a bitwise coding, we find a cost of6llog 2(|P|+|Q|)bits, which is non-trivial forl <|P|/12(ifPandQare approximately the same length).

So for all the metric distances described here, a single difference correcting message can be constructed which has sizeO(llogn), with low coefficients. They are all efficient with respect to their corresponding distances. Almost all of them are non-trivial schemes for up to a linear number of differences, which covers all practical situations. When there is a linear number of differences, then the two sequences will look very different indeed.

6.3.3

Computational Cost

While these protocols are communication-efficient, they are computationally expensive. This expense comes from the requirement to construct a graph of all possible objects. For example, for permutation edit distance, there are n! possible permutations of length n, so constructing the nodes of this graph requiresO(n!)time and space. Once the graph has been constructed the other operations are polynomial in the size of the graph; however, since the graph is exponentially large, these operations take time exponential inn. Firstly, G1 can be found as the graph with edges between each pair of

a b

Bit-string 1101 0100 0101 0011 1001 0100 0101 0011

Parity of whole string (locations 0-15) 0 1

Parity of locations 8,9,10,11,12,13,14,15 0 0

Parity of locations 4,5,6,7,12,13,14,15 1 1

Parity of locations 2,3,6,7,10,11,14,15 0 0

Parity of odd locations 0 1

A calculates the colour of the bitstringausing parities to get00100, which is sent to B. In turn, B finds the colour ofbas10101. By comparing the colours in a bitwise fashion, B can immediately find the location of the difference and hence work out what the stringais. The exclusive-or of the colours is10001. The first bit indicates thatais not identical tob; the next four bits can be read as a binary integer to show that the differing bit occurs at position 1.

Figure 6.1: Calculating a colour using Hamming codes

nodes that have a distance of one between their respective objects. In the permutation edit distance example, this can be done in timeO(n2n!), since each permutation is a distance one fromO(n2)others. ConstructingG2l fromG1can be done in time O((n!)2logl), and finally colouring this graph can be

done greedily in time linear in the size of the graph [Gib85]. So overall, the cost will be bounded by the most expensive operation, computingG2l, which isO(logl(n!)2). This is clearly impractical for all but

very small values ofn.

However, this technique is not solely of theoretical interest. Firstly, observe that since this procedure generates a single message, this message can be thought of as an error-correcting code to correct errors of the corresponding distance model. This proof shows that error-correcting codes for these models of error (transpositions, insertions, deletion, block operations etc.) exist, since we have shown concrete methods to construct such codes, albeit in a fashion that is computationally intractable. Secondly, note that the exponential cost of the method outlined is not an inherent part of constructing the message. Given a sequencea, we wish to give it a colour so that its colour is unique amongst all other objects within a distancedof another sequenceb. To ensure that both parties agreed on the colouring, we arranged for them both to construct the same graph from scratch and colour it using the same algorithm. What is instead desirable is if we could directly use a deterministic function to find the colour of an object. We would like that both the function and its inverse be computable in polynomial time. Then the overall computational cost would be polynomial rather than exponential. In fact, in some cases this is possible. In particular, the similarity of our colourings to error-correcting codes means that we can use techniques from the field of error-correcting codes to get our desired colourings. We show how this is possible for several of the distances under consideration.

Hamming Distance

The majority of coding theory focuses exclusively on the problem of correcting errors that change characters in place. It was precisely this situation that originally gave rise to the concept of the Hamming distance [Ham80]. We can use techniques from coding theory to efficiently compute the kinds of colours that are required. An example of this is given in Figure 6.1 which uses a Hamming code to allow the exchange of documents with up to a single difference. In this special case, the colour is calculated inO(nlogn)time, and requireslogn+ 1bits, which matches the lower bound in this case where h = 1and so is optimal. For the general case, Reed-Solomon codes can be used to create a colour for a string using2hlognbits to correct up tohdifferences. This is essentially the technique used in [AGE94]). We can extend this approach to certain other distances by taking advantage of their

embeddings into the Hamming distance.

Swap Distance

We saw in Section 3.2.1 that there is an exact, non-distortive embedding of swap distance into the Hamming distance. So for any given sequence, there is a corresponding binary matrix of size n2, and the Hamming distance between two matrices is the swap distance between the corresponding sequence. So to find the colour for a sequence under swap distance, we can compute its binary matrix, and then treat it as a binary string of lengthn2, and compute the colour accordingly. To correct up tohdifferences, a message of size2hlogn2 = 4hlognbits will suffice. One point to note is that swap distance can be as high asn(n−1)/2, which occurs between a permutation and its reverse. This protocol is only non-trivial forh < n/4.

Reversal Distance, Transposition Distance, RITE distances

These distances can also be embedded into the Hamming distance, although with distortive factors of between 2 and 3. Each permutation generates a binary matrix of sizen2bits. The same approach can be used as above: we treat the matrix as a bit-string of lengthn2, and use an error-correcting code to generate a colour. If the distance (reversal, transposition) isd, then the Hamming distance between the bit-strings will be betweendand2d. Hence we can use an error-correcting code to correct up to

2d differences. The cost of this will be2·2dlogn2 = 8dlognbits for transposition distances which is non-trivial for distances up ton/8. With a little care, the same can be achieved for RITE distances. The embedding of RITE distances is into theL1 metric; we can use the trivial embedding ofL1 into

Hamming distance (in Section 7), and so gain a single message cost of sizeO(dlogn).