A greedy algorithm, that in each step simply selects a random word that is not at distance < d to any of the words chosen so far thus will be able to make at least as
many choices as claimed. ◻
We call the set of words at bounded distance ≤ r the ball or (in coding theory) sphere of radius r
If d ≥ 2e + 1 these spheres of radius e may not overlap and thus give a similar upper bound:
Theorem IX.9 (Hamming bound, sphere-packing bound): Suppose that d ≥ 2e+1.
A q-ary code of length n and minimum distance d has at most
qn/ (∑e
i=0
(n
i)(q − 1)i) codewords.
If this bound is attained, that is the code has as many code words as possible, it is called a perfect code.
It is rare that a particular parameter set affords a perfect code.
Proposition IX.10 (Singleton bound): A q-ary code C of length n and minimum distance d has at most qn−d+1codewords.
Proof: Two words of C cannot agree in the first n−d +1 positions (as they then only would have d − 1 < d positions to differ). Thus codewords are determined by their entries in the first n − d + 1 positions, yielding qn−d+1possibilities. ◻ A code that achieves the Singleton bound is called maximum distance separable (MDS).
We note – Exercise ?? – that depending on the size of the alphabet the Hamming bound or the Singleton bound could be better.
There are further, better, bounds for the size of codes, but we shall not need these here.
IX.3 Linear Codes
So far we have not given any examples of codes apart from random selections of codewords. The first class of systematic constructions comes with tools from Lin-ear Algebra. This makes it easier to describe codes and will also provide a better decoding algorithm.
We now consider the case that q is a prime power and Q = Fq. In this case we can identify the Hamming space H(n, q) with the vector space Fnq.
Definition IX.11: A linear code is a subspace C ≤ Fnq. The weight of c ∈ C is the number wt(c) of nonzero entries of c. The minimum weight w of C is the minimum weight of nonzero c ∈ C.
If C is of dimension k, we call it an [n, k, w]-code.
Linearity means that distances are based on weights:
Lemma IX.12: Let C be linear and v, c ∈ C. Then d(v, c) = wt(v − c).
If C is a linear code, say of dimension k, it is the Fqspan of a basis, consisting of k vectors. We can write these vectors as rows of a k × n matrix G, called the generator matrix of C. (That is, C is the row space of G.) Note that the choice of G is not unique. Since we also can permute columns (replacing C with an equivalent code) one often assumes that G = (I ∣ P) for some matrix P.
Next we consider a basis of the null space (the vectors G ⋅ x = 0) of G and write the basis vectors as rows in a matrix H, that is GHT = 0. Then H is called a check matrix for C. Basic linear algebra gives us that G ∈ Fk×nq and H ∈ F(n−k)×nq , and that the rows of G are a basis of the null space of H.
Note that for w ∈ Fnq we have that w ∈ C if and only if wHT = 0, thus the term check matrix.
Generator matrix and check matrix now offer a convenient way of encoding and decoding. As the code is k-dimensional, the messages to be encoded will be simply vectors m ∈ Fkq. The encoding process then simply consists of the product m ⋅ G = c ∈ C. This is the code word that will be transmitted.
We receive a word w = c+u with u denoting the (as yet unknown) transmission error. For decoding w we calculate s = wHT = uHT(as cHT= 0 by definition). We call s the syndrome of w and note that it only depends in the error u. Furthermore, the map from error to syndrome is injective:
Suppose that u1, u2are two error vectors with u1HT = u2HT. Then (u1−u2)HT= 0, that is u1− u2is in the null space of H, that is in C.
On the other hand, since they are errors, we have that wt(u1), wt(u1) ≤ e and thus wt(u1− u2) ≤ 2e. The only vector in C of this weight is the zero vector.
This shows that in principle the error (and thus the correct decoding) can be determined from the syndrome, this is called syndrome decoding.
In some situations a clever choice of the check matrix H – e.g. putting it in RREF – allows for construction of an algorithm that computes u from s; in general however (as HTis not invertible), we might have to rely on a look-up table, listing errors for each syndrome.
Example: In a pathetic example, let Q = F2and
G =( 1 1 0 0 10 0 1 1 1 ) .
We easily find the four vectors in the row space of G and verify that this is a code of minimum weight 3, i.e. an [5, 2, 3] code. The message m = (1, 1) thus gets encoded by c = (1, 1, 1, 1, 0).
IX.3. LINEAR CODES 151 allowing us to recover the original codeword and message.
If two errors happened and we received w = c +u = (0, 1, 1, 0, 0). The syndrome then is [0, 1, 1], which could come from the errors (1, 0, 0, 1, 0) or (0, 1, 1, 0, 0).
Proposition IX.13: A linear code has minimum weight ≥ d if any d − 1 columns of its check matrix H are linearly dependent.
Proof: A product cHTis a linear combination of the columns of H with coefficients given by c. That is a linear relation amongst m columns of H corresponds to a word in C of weight m and vice versa. Since the minimum weight is ≥ d it is impossible
for < d columns to be linearly dependent. ◻
In the special case of 1-error correcting codes this simply means that any two columns of H must be linearly independent. For a given number d of rows, a max-imal set of such columns is obtained by representatives of the 1-dimensional sub-spaces of Fdq, corresponding to the [d1]
qpoints of PG(d − 1, q).
Definition IX.14: Let H be a d × [d1]qmatrix whose columns are representatives of PG(d − 1, q), and C the code of length n = [d1]qwhose check matrix is H. It is called the Hamming code over Fqof length n.
The hamming code of length n is a linear (n, n − d, 3) code.
Note IX.15: Changing representatives does not change C, changing the order of the columns will change the column order of C, an equivalent code.
Note IX.16: Syndrome decoding of Hamming codes is particularly easy. If the syn-drome s is nonzero, it is, up to a scalars α, equal to a unique (the i-th) column of H. The (weight one) error vector then is u = α ⋅ ei equals the third column of H, so the error was (indeed) in the third position.
Theorem IX.17: Hamming codes are perfect 1-error correcting.
Proof: The above discussion shows that a Hamming code is 1-error correcting and its length is n = [d1]q= (qd− 1)/(q − 1), so qd = n(q − 1) + 1. The number of code words is qn−d= qn/qd = qn/(1+n(q−1)), which is the Hamming bound for e = 1. ◻
TODO: Exercise, Lie